Skip to main content

PREDICCION DE INTERACIONES MULTISENSORIALES CON BASE EN LA ACTIVIDAD PRE-ESTIMULO

PSI2016-75558-P

•
Funding agency name Agencia Estatal de Investigación
Funding agency acronym AEI
Program Programa Estatal de Fomento de la Investigación Científica y Técnica de Excelencia
Subprogram Subprograma Estatal de Generación de Conocimiento
Call Proyectos de I+D (Excelencia / Generación de Conocimiento)
Call year 2016
State Plan 2013-2016 Plan Estatal de Investigación Científica y Técnica y de Innovación
Beneficiary centre UNIVERSITAT POMPEU FABRA CCT

Resultados relacionados

Resultados totales (Incluyendo duplicados): 29
Encontrada(s) 1 página(s)

Repositorio Digital de la UPF
Publication Artículo científico (JournalArticle)

Modality-switching in the Simon task: The clash of reference frames

  • Ruzzoli, Manuela
  • Soto-Faraco, Salvador, 1970-
The representation of spatial information related to an event can influence behavior even when location is task-irrelevant, as in the case of Stimulus–Response (S-R) compatibility effects on the Simon task. However, unlike single-modality situations, which are often used to study the Simon effect, in real-life scenarios various sensory modalities provide spatial information coded in different coordinate systems. Here, we address the expression of S-R compatibility effects in mixed-modality contexts, where events can occur in 1 of various sensory modalities (i.e., vision, touch or audition). The results confirm that, in single-modality cases, Simon effects in vision are expressed in an external spatial frame of reference, while touch information is coded anatomically. Remarkably, when mixing visual and tactile trials in an unpredictable way, the Simon effect disappeared in vision whereas tactile Simon effects remained expressed in their own (anatomical) frame of reference. Mixing visual and auditory stimuli did not obliterate the visual Simon effect and S-R compatibility effects in an external reference frame were evident for both modalities. The extinction of visual Simon effects as a result of mixing visual and tactile modalities can be interpreted as a consequence of the dynamic reorganization of the weights associated to the different sources of spatial information at play., This research was supported by the Ministerio de Economia y Competitividad (PSI2016-75558-P), the AGAUR Generalitat de Catalunya (2014SGR856) and the European Research Council (StG-2010 263145) to SSF. M.R. was supported by the Juan de la Cierva postdoctoral fellowship (JCI-2012-12335) from the Ministerio de Economia y Competitividad.
Repositorio Digital de la UPF
Publication Artículo científico (JournalArticle)

Can the occipital alpha-phase speed up visual detection through a real-time EEG-based brain-computer interface (BCI)?

  • Vigué Guix, Irene
  • Morís Fernández, Luis, 1982-
  • Torralba, Mireia
  • Ruzzoli, Manuela
  • Soto-Faraco, Salvador, 1970-
Electrical brain oscillations reflect fluctuations in neural excitability. Fluctuations in the alpha band (α, 8–12 Hz) in the occipito‐parietal cortex are thought to regulate sensory responses, leading to cyclic variations in visual perception. Inspired by this theory, some past and recent studies have addressed the relationship between α‐phase from extra‐cranial EEG and behavioural responses to visual stimuli in humans. The latest studies have used offline approaches to confirm α‐gated cyclic patterns. However, a particularly relevant implication is the possibility to use this principle online, whereby stimuli are time‐locked to specific α‐phases leading to predictable outcomes in performance. Here, we aimed at providing a proof of concept for such real‐time neurotechnology. Participants performed a speeded response task to visual targets that were presented upon a real‐time estimation of the α‐phase via an EEG closed‐loop brain–computer interface (BCI). According to the theory, we predicted a modulation of reaction times (RTs) along the α‐cycle. Our BCI system achieved reliable trial‐to‐trial phase locking of stimuli to the phase of individual occipito‐parietal α‐oscillations. Yet, the behavioural results did not support a consistent relation between RTs and the phase of the α‐cycle neither at group nor at single participant levels. We must conclude that although the α‐phase might play a role in perceptual decisions from a theoretical perspective, its impact on EEG‐based BCI application appears negligible., This research was supported by the Ministerio de Economia y Competitividad (PSI2016-75558-P AEI/FEDER to S.S.F.), AGAUR Generalitat de Catalunya (2014SGR856 to S.S.F.), Explora Ciencia 2015 (AEI - PSI2015-72568-EXP to M.R.). M.R. was also supported by the European Commission Individual Fellowship (Ctrl Code – 794649, H2020-MSCA-IF-2017).
Repositorio Digital de la UPF
Publication Artículo científico (JournalArticle)

Beat gestures and syntactic parsing: an ERP study

  • Biau, Emmanuel, 1985-
  • Fromont, Lauren A.
  • Soto-Faraco, Salvador, 1970-
We tested the prosodic hypothesis that the temporal alignment of a speaker's beat gestures in a sentence influences syntactic parsing by driving the listener's attention. Participants chose between two possible interpretations of relative-clause (RC) ambiguous sentences, while their electroencephalogram (EEG) was recorded. We manipulated the alignment of the beat within sentences where auditory prosody was removed. Behavioral performance showed no effect of beat placement on the sentences’ interpretation, while event-related potentials (ERPs) revealed a positive shift of the signal in the windows corresponding to N100 and P200 components. Additionally, post hoc analyses of the ERPs time locked to the RC revealed a modulation of the P600 component as a function of gesture. These results suggest that beats modulate early processing of affiliate words in continuous speech and potentially have a global impact at the level of sentence-parsing components. We speculate that beats must be synergistic with auditory prosody to be fully consequential in behavior., This research was supported by the Ministerio de Econom´ıa y Competitividad (PSI2016-75558-P), AGAUR Generalitat de Catalunya (2014SGR856), and the European Research Council (StG-2010 263145). EB was supported by a postdoctoral fellowship from the European Union’s Horizon 2020 research and innovation programme, under the Marie Sklodowska-Curie grant agreement No. 707727
Repositorio Digital de la UPF
Publication Artículo científico (JournalArticle)

Flexibility in reaction time analysis: many roads to a false positive?

  • Morís Fernández, Luis, 1982-
  • Vadillo, Miguel A.
In the present article, we explore the influence of undisclosed flexibility in the analysis of reaction times (RTs). RTs entail some degrees of freedom of their own, due to their skewed distribution, the potential presence of outliers and the availability of different methods to deal with these issues. Moreover, these degrees of freedom are usually not considered part of the analysis itself, but preprocessing steps that are contingent on data. We analysed the impact of these degrees of freedom on the false-positive rate using simulations over real and simulated data. When several preprocessing methods are used in combination, the false-positive rate can easily rise to 17%. This figure becomes more concerning if we consider that more degrees of freedom are awaiting down the analysis pipeline, potentially making the final false-positive rate much higher., The present work was supported by grant no. 2016-T1/SOC-1395 from Comunidad de Madrid (Programa de Atracción de Talento Investigador), grant nos. PSI2017-85159-P and PSI2016-75558-P from Agencia Estatal de Investigación (AEI, National Research Agency) and Fondo Europeo de Desarrollo Regional (FEDER, European Regional Development Fund), and grant no. 2017 SGR 1545 financed by the AGAUR (Catalan government).
Repositorio Digital de la UPF
Publication Artículo científico (JournalArticle)

Searching high and low: prosodic breaks disambiguate relative clauses

  • Fromont, Lauren A.
  • Soto-Faraco, Salvador, 1970-
  • Biau, Emmanuel, 1985-
During natural speech perception, listeners rely on a wide range of cues to support comprehension, from semantic context to prosodic information. There is a general consensus that prosody plays a role in syntactic parsing, but most studies focusing on ambiguous relative clauses (RC) show that prosodic cues, alone, are insufficient to reverse the preferred interpretation of sentence. These findings suggest that universally preferred structures (e.g., Late Closure principle) matter far more than prosodic cues in such cases. This study explores an alternative hypothesis: that the weak effect of prosody might be due to the influence of various syntactic, lexical-semantic, and acoustic confounding factors, and investigate the consequences of prosodic breaks while controlling these variables. We used Spanish RC sentences in three experimental conditions where the presence and position (following the first or second noun phrase) of prosodic breaks was manipulated. The results showed that the placement of a prosodic break determined sentence interpretation by changing the preferred attachment of the RC. Listeners' natural preference for low attachment (in the absence of break) was reinforced when a prosodic break was placed after the first noun. In contrast, a prosodic break placed after the second noun reversed the preferred interpretation of the sentence, toward high attachment. We argue that, in addition to other factors, listeners indeed use prosodic breaks as robust cues to syntactic parsing during speech processing, as these cues may direct listeners toward one interpretation or another., This research was supported by the Ministerio de Economía y Competitividad (PSI2016-75558-P), AGAUR Generalitat de Catalunya (2014SGR856), and the European Research Council (StG-2010 263145). EB was supported by a postdoctoral fellowship from the European Union’s Horizon 2020 research and innovation programme, under the Marie Sklodowska-Curie grant agreement No. 707727.
Repositorio Digital de la UPF
Publication Artículo científico (JournalArticle)

Beliefs about others’ intentions determine whether cooperation is the faster choice

  • Castro Santa, Juana
  • Exadaktylos, Filippos
  • Soto-Faraco, Salvador, 1970-
Is collaboration the fast choice for humans? Past studies proposed that cooperation is a behavioural default, based on Response Times (RT) findings. Here we contend that the individual’s reckoning of the immediate social environment shapes her predisposition to cooperate and, hence, response latencies. In a social dilemma game, we manipulate the beliefs about the partner’s intentions to cooperate and show that they act as a switch that determines cooperation and defection RTs; when the partner’s intention to cooperate is perceived as high, cooperation choices are speeded up, while defection is slowed down. Importantly, this social context effect holds across varying expected payoffs, indicating that it modulates behaviour regardless of choices’ similarity in monetary terms. Moreover, this pattern is moderated by individual variability in social preferences: Among conditional cooperators, high cooperation beliefs speed up cooperation responses and slow down defection. Among free-riders, defection is always faster and more likely than cooperation, while high cooperation beliefs slow down all decisions. These results shed new light on the conflict of choices account of response latencies, as well as on the intuitive cooperation hypothesis, and can help to correctly interpret and reconcile previous, apparently contradictory results, by considering the role of context in social dilemmas., SS-F was supported by the Ministerio de Economía y Competitividad (PSI2016-75558-P AEI/FEDER) and AGAUR Generalitat de Catalunya (2017 SGR 1545). FE acknowledges the European Commission Marie Curie postdoctoral fellowship (H2020-MSCA-IF-2014, Grant number: 657741).
Repositorio Digital de la UPF
Publication Artículo científico (JournalArticle)

The breakdown of the Simon effect in cross‐modal contexts: EEG evidence

  • Castro, Leonor
  • Soto-Faraco, Salvador, 1970-
  • Morís Fernández, Luis, 1982-
  • Ruzzoli, Manuela
In everyday life, we often must coordinate information across spatial locations and different senses for action. It is well known, for example, that reactions are faster when an imperative stimulus and its required response are congruent than when they are not, even if stimulus location itself is completely irrelevant for the task (the so-called Simon effect). However, because these effects have been frequently investigated in single-modality scenarios, the consequences of spatial congruence when more than one sensory modality is at play are less well known. Interestingly, at a behavioral level, the visual Simon effect vanishes in mixed (visual and tactile) modality scenarios, suggesting that irrelevant spatial information ceases to exert influence on vision. To shed some light on this surprising result, here we address the expression of irrelevant spatial information in EEG markers typical of the visual Simon effect (P300, theta power modulation, LRP) in mixed-modality contexts. Our results show no evidence for the visual- spatial information to affect performance at behavioral and neurophysiological levels. The absence of evidence of the neural markers of visual S-R conflict in the mixed-modality scenario implies that some aspects of spatial representations that are strongly expressed in single-modality scenarios might be bypassed., This research was supported by the Ministerio de Economia, Industria y Competividad (AEI/FEDER‐PSI2016‐75558‐P to SSF; and Explora Ciencia 2015 PSI2015‐72568‐EXP‐AEI to MR), and AGAUR Generalitat de Catalunya (2014SGR856 to SSF).
Repositorio Digital de la UPF
Publication Artículo científico (JournalArticle)

From cognitive control to visual incongruity: conflict detection in surrealistic images

  • Ruzzoli, Manuela
  • McGuinness, Aoife
  • Morís Fernández, Luis, 1982-
  • Soto-Faraco, Salvador, 1970-
This study explored brain responses to images that exploit incongruity as a creative technique, often used in advertising (i.e., surrealistic images). We hypothesized that these images would reveal responses akin to cognitive conflict resulting from incongruent trials in typical laboratory tasks (i.e., Stroop Task). Indeed, in many surrealistic images, common visual elements are juxtaposed to create un-ordinary associations with semantically conflicting representations. We expected that these images engage the conflict processing network that has been described in cognitive neuroscience theories. We addressed this hypothesis by measuring the power of mid-frontal Theta oscillations using EEG while participants watched images through a social media-like interface. Incongruent images, compared to controls, produced a significant Theta power increase, as predicted from the cognitive conflict theory. We also found increased memory for incongruent images one week after exposure, compared to the controls. These findings provide evidence for the incongruent images to effectively engage the viewer’s cognitive control and boost memorability. The results of this study provide validation of cognitive theories in real-life scenarios (i.e., surrealistic ads or art) and offer insights regarding the use of neural correlates as effectiveness metrics in advertising., This research was supported by the Ministerio de Economia y Competitividad (PSI2016-75558-P AEI/FEDER), AGAUR Generalitat de Catalunya (2017 SGR 1545), and the European Research Council (PoC- 727595 to SSF) and. M.R. was supported by a Marie Skłodowska-Curie fellowship (Ctrl Code – 794649 - H2020-MSCA-IF-2017 to MR).
Repositorio Digital de la UPF
Publication Artículo científico (JournalArticle)

Sounds can boost the awareness of visual events through attention without cross-modal integration

  • Soto-Faraco, Salvador, 1970-
  • Pápai, Márta Szabina, 1987-
Cross-modal interactions can lead to enhancement of visual perception, even for visual events below awareness. However, the underlying mechanism is still unclear. Can purely bottom-up cross-modal integration break through the threshold of awareness? We used a binocular rivalry paradigm to measure perceptual switches after brief flashes or sounds which, sometimes, co-occurred. When flashes at the suppressed eye coincided with sounds, perceptual switches occurred the earliest. Yet, contrary to the hypothesis of cross-modal integration, this facilitation never surpassed the assumption of probability summation of independent sensory signals. A follow-up experiment replicated the same pattern of results using silent gaps embedded in continuous noise, instead of sounds. This manipulation should weaken putative sound-flash integration, although keep them salient as bottom-up attention cues. Additional results showed that spatial congruency between flashes and sounds did not determine the effectiveness of cross-modal facilitation, which was again not better than probability summation. Thus, the present findings fail to fully support the hypothesis of bottom-up cross-modal integration, above and beyond the independent contribution of two transient signals, as an account for cross-modal enhancement of visual events below level of awareness., This study was funded by a Ministerio de Educación, Cultura y Deporte, Ref. FPU/1206471 to MSP, and ERC Starting Grant, Ref. 263145, Spanish Ministry of Science and Innovation (PSI2016-75558-P), and Comissionat per a Universitats i Recerca del DIUE- Generalitat de Catalunya (2014SGR856) to SS-F.
Repositorio Digital de la UPF
Publication Artículo científico (JournalArticle)

Theta oscillations reflect conflict processing in the perception of the McGurk illusion

  • Morís Fernández, Luis, 1982-
  • Torralba, Mireia
  • Soto-Faraco, Salvador, 1970-
The McGurk illusion is one of the most famous illustrations of cross‐modal integration in human perception. It has been often used as a proxy of audiovisual (AV) integration and to infer the properties of the integration process in natural (congruent) AV conditions. Nonetheless, a blatant difference between McGurk stimuli and natural, congruent, AV speech is the conflict between the auditory and the visual information in the former. Here, we hypothesized that McGurk stimuli (and any AV incongruency) engage brain responses similar to those found in more general cases of perceptual conflict (e.g., Stroop), and propose that the McGurk illusion arises as a result of the resolution of such conflict. We used electroencephalography to measure variations in the power of theta, a well‐known marker of the brain response to conflict. The results showed that perception of AV McGurk stimuli, just like AV incongruence in general, induces an increase in activity in the theta band. This response was similar to that evoked by Stroop stimuli, as measured in the same participants. This finding suggests that the McGurk illusion is mediated by general‐purpose conflict mechanisms, and calls for caution in generalizing findings obtained using the McGurk illusion, to the general case of multisensory integration., This research was supported by the Ministerio de Economia y Competitividad (PSI2016‐75558‐P AEI/FEDER), AGAUR Generalitat de Catalunya (2014SGR856) and the European Research Council (StG‐2010 263145).
Repositorio Digital de la UPF
Publication Artículo científico (JournalArticle)

Characteristic sounds facilitate object search in real-life scenes

  • Kvasova, Daria, 1989-
  • Garcia-Vernet, Laia
  • Soto-Faraco, Salvador, 1970-
Real-world events do not only provide temporally and spatially correlated information across the senses, but also semantic correspondences about object identity. Prior research has shown that object sounds can enhance detection, identification, and search performance of semantically consistent visual targets. However, these effects are always demonstrated in simple and stereotyped displays that lack ecological validity. In order to address identity-based cross-modal relationships in real-world scenarios, we designed a visual search task using complex, dynamic scenes. Participants searched for objects in video clips recorded from real-life scenes. Auditory cues, embedded in the background sounds, could be target-consistent, distracter-consistent, neutral, or just absent. We found that, in these naturalistic scenes, characteristic sounds improve visual search for task-relevant objects but fail to increase the salience of irrelevant distracters. Our findings generalize previous results on object-based cross-modal interactions with simple stimuli and shed light upon how audio–visual semantically congruent relationships play out in real-life contexts., This research was supported by the Ministerio de Economia y Competitividad (PSI2016-75558-P AEI/FEDER) and the AGAUR Generalitat de Catalunya (2017 SGR 1545). DK was supported by an FI scholarship, from the AGAUR Generalitat de Catalunya. This manuscript has been released as a pre-print at bioRxiv (Kvasova et al., 2019).
Repositorio Digital de la UPF
Publication Artículo científico (JournalArticle)

The Dynamics of decision-making and action during active sampling

  • Ozbagci, Duygu
  • Moreno Bote, Rubén
  • Soto-Faraco, Salvador, 1970-
Embodied Cognition Theories (ECTs) of decision-making propose that the decision process pervades the execution of choice actions and manifests itself in these actions. Decision-making scenarios where actions not only express the choice but also help sample information can provide a valuable, ecologically relevant model for this framework. We present a study to address this paradigmatic situation in humans. Subjects categorized (2AFC task) a central object image, blurred to diferent extents, by moving a cursor toward the left or right of the display. Upward cursor movements reduced the image blur and could be used to sample information. Thus, actions for decision and actions for sampling were orthogonal to each other. We analyzed response trajectories to test whether information-sampling movements co-occurred with the ongoing decision process. Trajectories were bimodally distributed, with one kind being direct towards one response option (non-sampling), and the other kind containing an initial upward component before veering of towards an option (sampling). This implies that there was an initial decision at the early stage of a trial, whether to sample information or not. Importantly, in sampling trials trajectories were not purely upward, but rather had a signifcant horizontal deviation early on. This result suggests that movements to sample information exhibit an online interaction with the decision process, therefore supporting the prediction of the ECTs under ecologically relevant constrains., This project receives support from the grant from Agency for Management of University and Research Grants (AGAUR-2017 SGR 1545), Ministerio de Economia y Competitividad (PSI2016-75558-P AEI/FEDER). Tis project has been co-funded with 50% by the European Regional Development Fund under the framework of the ERFD/FEDER Operative Programme for Catalunya 2014–2020. D.O. is supported by a fellowship from “la Caixa” Foundation (ID100010434; fellowship code LCF/BQ/DI18/11660026). Tis project has received funding from the European Union’s Horizon 2020 research and innovation programme under the Marie Skłodowska-Curie grant agreement No. 713673. R.M.B. is supported by BFU2017-85936-P from MINECO (Spain), the Howard Hughes Medical Institute (HHMI; ref 55008742), an ICREA Academia award, and the Bial Foundation (grant number 117/18). S.S.F. is funded by Ministerio de Ciencia e Innovación (Ref: PID2019-108531GB-I00 AEI/FEDER).
Repositorio Digital de la UPF
Publication Artículo científico (JournalArticle)

Audiovisual integration as conflict resolution: The conflict of the McGurk illusion

  • Morís Fernández, Luis, 1982-
  • Macaluso, Emiliano
  • Soto-Faraco, Salvador, 1970-
There are two main behavioral expressions of multisensory integration (MSI) in speech; the perceptual enhancement produced by the sight of the congruent lip movements of the speaker, and the illusory sound perceived when a speech syllable is dubbed with incongruent lip movements, in the McGurk effect. These two models have been used very often to study MSI. Here, we contend that, unlike congruent audiovisually (AV) speech, the McGurk effect involves brain areas related to conflict detection and resolution. To test this hypothesis, we used fMRI to measure blood oxygen level dependent responses to AV speech syllables. We analyzed brain activity as a function of the nature of the stimuli—McGurk or non‐McGurk—and the perceptual outcome regarding MSI—integrated or not integrated response—in a 2 × 2 factorial design. The results showed that, regardless of perceptual outcome, AV mismatch activated general‐purpose conflict areas (e.g., anterior cingulate cortex) as well as specific AV speech conflict areas (e.g., inferior frontal gyrus), compared with AV matching stimuli. Moreover, these conflict areas showed stronger activation on trials where the McGurk illusion was perceived compared with non‐illusory trials, despite the stimuli where physically identical. We conclude that the AV incongruence in McGurk stimuli triggers the activation of conflict processing areas and that the process of resolving the cross‐modal conflict is critical for the McGurk illusion to arise. Hum Brain Mapp 38:5691–5705, 2017., This research was supported by the Ministerio de Economía y Competitividad (PSI2016–75558‐PAEI/FEDER), AGAUR Generalitat de Catalunya (2014SGR856 and 2012BE100392), and the European Research Council (StG‐2010 263145). The Neuroimaging Laboratory of the Fondazione Santa Lucia is supported by the Italian Ministry of Health. EM is supported by the program “Investissements d'Avenir” (ANR‐11‐IDEX‐0007) and by the program “BQR Accueil EC 16” of University Claude Bernard Lyon 1.
Repositorio Digital de la UPF
Publication Artículo científico (JournalArticle)

Cross-modal decoupling in temporal attention between audition and touch

  • Mühlberg, Stefanie, 1986-
  • Soto-Faraco, Salvador, 1970-
Temporal orienting leads to well-documented behavioural benefits for sensory events occurring at the anticipated moment. However, the consequences of temporal orienting in cross-modal contexts are still unclear. On the one hand, some studies using audio-tactile paradigms suggest that attentional orienting in time and modality are a closely coupled system, in which temporal orienting dominates modality orienting, similar to what happens in cross-modal spatial attention. On the other hand, recent findings using a visuo-tactile paradigm suggest that attentional orienting in time can unfold independently in each modality, leading to cross-modal decoupling. In the present study, we investigated if cross-modal decoupling in time can be extrapolated to audio-tactile contexts. If so, decoupling might represent a general property of cross-modal attention in time. To this end, we used a speeded discrimination task in which we manipulated the probability of target presentation in time and modality. In each trial, a manipulation of time-based expectancy was used to guide participants’ attention to task-relevant events, either tactile or auditory, at different points in time. In two experiments, we found that participants generally showed enhanced behavioural performance at the most likely onset time of each modality and no evidence for coupling. This pattern supports the hypothesis that cross-modal decoupling could be a general phenomenon in temporal orienting., This research was funded by the Spanish Ministry of Science and Innovation (PSI2016-75558-P(AEI/FEDER)), the Comissionat per a Universitats i Recerca del DIUE-Generalitat de Catalunya (2014SGR856), and the European Research Council (StG-2010 263145) to S.S.F.
Recercat. Dipósit de la Recerca de Catalunya
Publication Artículo científico (JournalArticle)

Theta oscillations reflect conflict processing in the perception of the McGurk illusion

  • Morís Fernández, Luis, 1982-
  • Torralba, Mireia
  • Soto-Faraco, Salvador, 1970-
The McGurk illusion is one of the most famous illustrations of cross‐modal integration in human perception. It has been often used as a proxy of audiovisual (AV) integration and to infer the properties of the integration process in natural (congruent) AV conditions. Nonetheless, a blatant difference between McGurk stimuli and natural, congruent, AV speech is the conflict between the auditory and the visual information in the former. Here, we hypothesized that McGurk stimuli (and any AV incongruency) engage brain responses similar to those found in more general cases of perceptual conflict (e.g., Stroop), and propose that the McGurk illusion arises as a result of the resolution of such conflict. We used electroencephalography to measure variations in the power of theta, a well‐known marker of the brain response to conflict. The results showed that perception of AV McGurk stimuli, just like AV incongruence in general, induces an increase in activity in the theta band. This response was similar to that evoked by Stroop stimuli, as measured in the same participants. This finding suggests that the McGurk illusion is mediated by general‐purpose conflict mechanisms, and calls for caution in generalizing findings obtained using the McGurk illusion, to the general case of multisensory integration., This research was supported by the Ministerio de Economia y Competitividad (PSI2016‐75558‐P AEI/FEDER), AGAUR Generalitat de Catalunya (2014SGR856) and the European Research Council (StG‐2010 263145).
Recercat. Dipósit de la Recerca de Catalunya
Publication Artículo científico (JournalArticle)

Audiovisual integration as conflict resolution: The conflict of the McGurk illusion

  • Morís Fernández, Luis, 1982-
  • Macaluso, Emiliano
  • Soto-Faraco, Salvador, 1970-
There are two main behavioral expressions of multisensory integration (MSI) in speech; the perceptual enhancement produced by the sight of the congruent lip movements of the speaker, and the illusory sound perceived when a speech syllable is dubbed with incongruent lip movements, in the McGurk effect. These two models have been used very often to study MSI. Here, we contend that, unlike congruent audiovisually (AV) speech, the McGurk effect involves brain areas related to conflict detection and resolution. To test this hypothesis, we used fMRI to measure blood oxygen level dependent responses to AV speech syllables. We analyzed brain activity as a function of the nature of the stimuli—McGurk or non‐McGurk—and the perceptual outcome regarding MSI—integrated or not integrated response—in a 2 × 2 factorial design. The results showed that, regardless of perceptual outcome, AV mismatch activated general‐purpose conflict areas (e.g., anterior cingulate cortex) as well as specific AV speech conflict areas (e.g., inferior frontal gyrus), compared with AV matching stimuli. Moreover, these conflict areas showed stronger activation on trials where the McGurk illusion was perceived compared with non‐illusory trials, despite the stimuli where physically identical. We conclude that the AV incongruence in McGurk stimuli triggers the activation of conflict processing areas and that the process of resolving the cross‐modal conflict is critical for the McGurk illusion to arise. Hum Brain Mapp 38:5691–5705, 2017., This research was supported by the Ministerio de Economía y Competitividad (PSI2016–75558‐PAEI/FEDER), AGAUR Generalitat de Catalunya (2014SGR856 and 2012BE100392), and the European Research Council (StG‐2010 263145). The Neuroimaging Laboratory of the Fondazione Santa Lucia is supported by the Italian Ministry of Health. EM is supported by the program “Investissements d'Avenir” (ANR‐11‐IDEX‐0007) and by the program “BQR Accueil EC 16” of University Claude Bernard Lyon 1.
Recercat. Dipósit de la Recerca de Catalunya
Publication Artículo científico (JournalArticle)

Searching high and low: prosodic breaks disambiguate relative clauses

  • Fromont, Lauren A.
  • Soto-Faraco, Salvador, 1970-
  • Biau, Emmanuel, 1985-
During natural speech perception, listeners rely on a wide range of cues to support comprehension, from semantic context to prosodic information. There is a general consensus that prosody plays a role in syntactic parsing, but most studies focusing on ambiguous relative clauses (RC) show that prosodic cues, alone, are insufficient to reverse the preferred interpretation of sentence. These findings suggest that universally preferred structures (e.g., Late Closure principle) matter far more than prosodic cues in such cases. This study explores an alternative hypothesis: that the weak effect of prosody might be due to the influence of various syntactic, lexical-semantic, and acoustic confounding factors, and investigate the consequences of prosodic breaks while controlling these variables. We used Spanish RC sentences in three experimental conditions where the presence and position (following the first or second noun phrase) of prosodic breaks was manipulated. The results showed that the placement of a prosodic break determined sentence interpretation by changing the preferred attachment of the RC. Listeners' natural preference for low attachment (in the absence of break) was reinforced when a prosodic break was placed after the first noun. In contrast, a prosodic break placed after the second noun reversed the preferred interpretation of the sentence, toward high attachment. We argue that, in addition to other factors, listeners indeed use prosodic breaks as robust cues to syntactic parsing during speech processing, as these cues may direct listeners toward one interpretation or another., This research was supported by the Ministerio de Economía y Competitividad (PSI2016-75558-P), AGAUR Generalitat de Catalunya (2014SGR856), and the European Research Council (StG-2010 263145). EB was supported by a postdoctoral fellowship from the European Union’s Horizon 2020 research and innovation programme, under the Marie Sklodowska-Curie grant agreement No. 707727.
Recercat. Dipósit de la Recerca de Catalunya
Publication Artículo científico (JournalArticle)

Sounds can boost the awareness of visual events through attention without cross-modal integration

  • Soto-Faraco, Salvador, 1970-
  • Pápai, Márta Szabina, 1987-
Cross-modal interactions can lead to enhancement of visual perception, even for visual events below awareness. However, the underlying mechanism is still unclear. Can purely bottom-up cross-modal integration break through the threshold of awareness? We used a binocular rivalry paradigm to measure perceptual switches after brief flashes or sounds which, sometimes, co-occurred. When flashes at the suppressed eye coincided with sounds, perceptual switches occurred the earliest. Yet, contrary to the hypothesis of cross-modal integration, this facilitation never surpassed the assumption of probability summation of independent sensory signals. A follow-up experiment replicated the same pattern of results using silent gaps embedded in continuous noise, instead of sounds. This manipulation should weaken putative sound-flash integration, although keep them salient as bottom-up attention cues. Additional results showed that spatial congruency between flashes and sounds did not determine the effectiveness of cross-modal facilitation, which was again not better than probability summation. Thus, the present findings fail to fully support the hypothesis of bottom-up cross-modal integration, above and beyond the independent contribution of two transient signals, as an account for cross-modal enhancement of visual events below level of awareness., This study was funded by a Ministerio de Educación, Cultura y Deporte, Ref. FPU/1206471 to MSP, and ERC Starting Grant, Ref. 263145, Spanish Ministry of Science and Innovation (PSI2016-75558-P), and Comissionat per a Universitats i Recerca del DIUE- Generalitat de Catalunya (2014SGR856) to SS-F.
Recercat. Dipósit de la Recerca de Catalunya
Publication Artículo científico (JournalArticle)

Modality-switching in the Simon task: The clash of reference frames

  • Ruzzoli, Manuela
  • Soto-Faraco, Salvador, 1970-
The representation of spatial information related to an event can influence behavior even when location is task-irrelevant, as in the case of Stimulus–Response (S-R) compatibility effects on the Simon task. However, unlike single-modality situations, which are often used to study the Simon effect, in real-life scenarios various sensory modalities provide spatial information coded in different coordinate systems. Here, we address the expression of S-R compatibility effects in mixed-modality contexts, where events can occur in 1 of various sensory modalities (i.e., vision, touch or audition). The results confirm that, in single-modality cases, Simon effects in vision are expressed in an external spatial frame of reference, while touch information is coded anatomically. Remarkably, when mixing visual and tactile trials in an unpredictable way, the Simon effect disappeared in vision whereas tactile Simon effects remained expressed in their own (anatomical) frame of reference. Mixing visual and auditory stimuli did not obliterate the visual Simon effect and S-R compatibility effects in an external reference frame were evident for both modalities. The extinction of visual Simon effects as a result of mixing visual and tactile modalities can be interpreted as a consequence of the dynamic reorganization of the weights associated to the different sources of spatial information at play., This research was supported by the Ministerio de Economia y Competitividad (PSI2016-75558-P), the AGAUR Generalitat de Catalunya (2014SGR856) and the European Research Council (StG-2010 263145) to SSF. M.R. was supported by the Juan de la Cierva postdoctoral fellowship (JCI-2012-12335) from the Ministerio de Economia y Competitividad.
Recercat. Dipósit de la Recerca de Catalunya
Publication Artículo científico (JournalArticle)

The Dynamics of decision-making and action during active sampling

  • Ozbagci, Duygu
  • Moreno Bote, Rubén
  • Soto-Faraco, Salvador, 1970-
Embodied Cognition Theories (ECTs) of decision-making propose that the decision process pervades the execution of choice actions and manifests itself in these actions. Decision-making scenarios where actions not only express the choice but also help sample information can provide a valuable, ecologically relevant model for this framework. We present a study to address this paradigmatic situation in humans. Subjects categorized (2AFC task) a central object image, blurred to diferent extents, by moving a cursor toward the left or right of the display. Upward cursor movements reduced the image blur and could be used to sample information. Thus, actions for decision and actions for sampling were orthogonal to each other. We analyzed response trajectories to test whether information-sampling movements co-occurred with the ongoing decision process. Trajectories were bimodally distributed, with one kind being direct towards one response option (non-sampling), and the other kind containing an initial upward component before veering of towards an option (sampling). This implies that there was an initial decision at the early stage of a trial, whether to sample information or not. Importantly, in sampling trials trajectories were not purely upward, but rather had a signifcant horizontal deviation early on. This result suggests that movements to sample information exhibit an online interaction with the decision process, therefore supporting the prediction of the ECTs under ecologically relevant constrains., This project receives support from the grant from Agency for Management of University and Research Grants (AGAUR-2017 SGR 1545), Ministerio de Economia y Competitividad (PSI2016-75558-P AEI/FEDER). Tis project has been co-funded with 50% by the European Regional Development Fund under the framework of the ERFD/FEDER Operative Programme for Catalunya 2014–2020. D.O. is supported by a fellowship from “la Caixa” Foundation (ID100010434; fellowship code LCF/BQ/DI18/11660026). Tis project has received funding from the European Union’s Horizon 2020 research and innovation programme under the Marie Skłodowska-Curie grant agreement No. 713673. R.M.B. is supported by BFU2017-85936-P from MINECO (Spain), the Howard Hughes Medical Institute (HHMI; ref 55008742), an ICREA Academia award, and the Bial Foundation (grant number 117/18). S.S.F. is funded by Ministerio de Ciencia e Innovación (Ref: PID2019-108531GB-I00 AEI/FEDER).
Recercat. Dipósit de la Recerca de Catalunya
Publication Artículo científico (JournalArticle)

The breakdown of the Simon effect in cross‐modal contexts: EEG evidence

  • Castro, Leonor
  • Soto-Faraco, Salvador, 1970-
  • Morís Fernández, Luis, 1982-
  • Ruzzoli, Manuela
In everyday life, we often must coordinate information across spatial locations and different senses for action. It is well known, for example, that reactions are faster when an imperative stimulus and its required response are congruent than when they are not, even if stimulus location itself is completely irrelevant for the task (the so-called Simon effect). However, because these effects have been frequently investigated in single-modality scenarios, the consequences of spatial congruence when more than one sensory modality is at play are less well known. Interestingly, at a behavioral level, the visual Simon effect vanishes in mixed (visual and tactile) modality scenarios, suggesting that irrelevant spatial information ceases to exert influence on vision. To shed some light on this surprising result, here we address the expression of irrelevant spatial information in EEG markers typical of the visual Simon effect (P300, theta power modulation, LRP) in mixed-modality contexts. Our results show no evidence for the visual- spatial information to affect performance at behavioral and neurophysiological levels. The absence of evidence of the neural markers of visual S-R conflict in the mixed-modality scenario implies that some aspects of spatial representations that are strongly expressed in single-modality scenarios might be bypassed., This research was supported by the Ministerio de Economia, Industria y Competividad (AEI/FEDER‐PSI2016‐75558‐P to SSF; and Explora Ciencia 2015 PSI2015‐72568‐EXP‐AEI to MR), and AGAUR Generalitat de Catalunya (2014SGR856 to SSF).
Recercat. Dipósit de la Recerca de Catalunya
Publication Artículo científico (JournalArticle)

Flexibility in reaction time analysis: many roads to a false positive?

  • Morís Fernández, Luis, 1982-
  • Vadillo, Miguel A.
In the present article, we explore the influence of undisclosed flexibility in the analysis of reaction times (RTs). RTs entail some degrees of freedom of their own, due to their skewed distribution, the potential presence of outliers and the availability of different methods to deal with these issues. Moreover, these degrees of freedom are usually not considered part of the analysis itself, but preprocessing steps that are contingent on data. We analysed the impact of these degrees of freedom on the false-positive rate using simulations over real and simulated data. When several preprocessing methods are used in combination, the false-positive rate can easily rise to 17%. This figure becomes more concerning if we consider that more degrees of freedom are awaiting down the analysis pipeline, potentially making the final false-positive rate much higher., The present work was supported by grant no. 2016-T1/SOC-1395 from Comunidad de Madrid (Programa de Atracción de Talento Investigador), grant nos. PSI2017-85159-P and PSI2016-75558-P from Agencia Estatal de Investigación (AEI, National Research Agency) and Fondo Europeo de Desarrollo Regional (FEDER, European Regional Development Fund), and grant no. 2017 SGR 1545 financed by the AGAUR (Catalan government).
Recercat. Dipósit de la Recerca de Catalunya
Publication Artículo científico (JournalArticle)

Can the occipital alpha-phase speed up visual detection through a real-time EEG-based brain-computer interface (BCI)?

  • Vigué Guix, Irene
  • Morís Fernández, Luis, 1982-
  • Torralba, Mireia
  • Ruzzoli, Manuela
  • Soto-Faraco, Salvador, 1970-
Electrical brain oscillations reflect fluctuations in neural excitability. Fluctuations in the alpha band (α, 8–12 Hz) in the occipito‐parietal cortex are thought to regulate sensory responses, leading to cyclic variations in visual perception. Inspired by this theory, some past and recent studies have addressed the relationship between α‐phase from extra‐cranial EEG and behavioural responses to visual stimuli in humans. The latest studies have used offline approaches to confirm α‐gated cyclic patterns. However, a particularly relevant implication is the possibility to use this principle online, whereby stimuli are time‐locked to specific α‐phases leading to predictable outcomes in performance. Here, we aimed at providing a proof of concept for such real‐time neurotechnology. Participants performed a speeded response task to visual targets that were presented upon a real‐time estimation of the α‐phase via an EEG closed‐loop brain–computer interface (BCI). According to the theory, we predicted a modulation of reaction times (RTs) along the α‐cycle. Our BCI system achieved reliable trial‐to‐trial phase locking of stimuli to the phase of individual occipito‐parietal α‐oscillations. Yet, the behavioural results did not support a consistent relation between RTs and the phase of the α‐cycle neither at group nor at single participant levels. We must conclude that although the α‐phase might play a role in perceptual decisions from a theoretical perspective, its impact on EEG‐based BCI application appears negligible., This research was supported by the Ministerio de Economia y Competitividad (PSI2016-75558-P AEI/FEDER to S.S.F.), AGAUR Generalitat de Catalunya (2014SGR856 to S.S.F.), Explora Ciencia 2015 (AEI - PSI2015-72568-EXP to M.R.). M.R. was also supported by the European Commission Individual Fellowship (Ctrl Code – 794649, H2020-MSCA-IF-2017).
Recercat. Dipósit de la Recerca de Catalunya
Publication Artículo científico (JournalArticle)

Beliefs about others’ intentions determine whether cooperation is the faster choice

  • Castro Santa, Juana
  • Exadaktylos, Filippos
  • Soto-Faraco, Salvador, 1970-
Is collaboration the fast choice for humans? Past studies proposed that cooperation is a behavioural default, based on Response Times (RT) findings. Here we contend that the individual’s reckoning of the immediate social environment shapes her predisposition to cooperate and, hence, response latencies. In a social dilemma game, we manipulate the beliefs about the partner’s intentions to cooperate and show that they act as a switch that determines cooperation and defection RTs; when the partner’s intention to cooperate is perceived as high, cooperation choices are speeded up, while defection is slowed down. Importantly, this social context effect holds across varying expected payoffs, indicating that it modulates behaviour regardless of choices’ similarity in monetary terms. Moreover, this pattern is moderated by individual variability in social preferences: Among conditional cooperators, high cooperation beliefs speed up cooperation responses and slow down defection. Among free-riders, defection is always faster and more likely than cooperation, while high cooperation beliefs slow down all decisions. These results shed new light on the conflict of choices account of response latencies, as well as on the intuitive cooperation hypothesis, and can help to correctly interpret and reconcile previous, apparently contradictory results, by considering the role of context in social dilemmas., SS-F was supported by the Ministerio de Economía y Competitividad (PSI2016-75558-P AEI/FEDER) and AGAUR Generalitat de Catalunya (2017 SGR 1545). FE acknowledges the European Commission Marie Curie postdoctoral fellowship (H2020-MSCA-IF-2014, Grant number: 657741).
Recercat. Dipósit de la Recerca de Catalunya
Publication Artículo científico (JournalArticle)

From cognitive control to visual incongruity: conflict detection in surrealistic images

  • Ruzzoli, Manuela
  • McGuinness, Aoife
  • Morís Fernández, Luis, 1982-
  • Soto-Faraco, Salvador, 1970-
This study explored brain responses to images that exploit incongruity as a creative technique, often used in advertising (i.e., surrealistic images). We hypothesized that these images would reveal responses akin to cognitive conflict resulting from incongruent trials in typical laboratory tasks (i.e., Stroop Task). Indeed, in many surrealistic images, common visual elements are juxtaposed to create un-ordinary associations with semantically conflicting representations. We expected that these images engage the conflict processing network that has been described in cognitive neuroscience theories. We addressed this hypothesis by measuring the power of mid-frontal Theta oscillations using EEG while participants watched images through a social media-like interface. Incongruent images, compared to controls, produced a significant Theta power increase, as predicted from the cognitive conflict theory. We also found increased memory for incongruent images one week after exposure, compared to the controls. These findings provide evidence for the incongruent images to effectively engage the viewer’s cognitive control and boost memorability. The results of this study provide validation of cognitive theories in real-life scenarios (i.e., surrealistic ads or art) and offer insights regarding the use of neural correlates as effectiveness metrics in advertising., This research was supported by the Ministerio de Economia y Competitividad (PSI2016-75558-P AEI/FEDER), AGAUR Generalitat de Catalunya (2017 SGR 1545), and the European Research Council (PoC- 727595 to SSF) and. M.R. was supported by a Marie Skłodowska-Curie fellowship (Ctrl Code – 794649 - H2020-MSCA-IF-2017 to MR).
Recercat. Dipósit de la Recerca de Catalunya
Publication Artículo científico (JournalArticle)

Cross-modal decoupling in temporal attention between audition and touch

  • Mühlberg, Stefanie, 1986-
  • Soto-Faraco, Salvador, 1970-
Temporal orienting leads to well-documented behavioural benefits for sensory events occurring at the anticipated moment. However, the consequences of temporal orienting in cross-modal contexts are still unclear. On the one hand, some studies using audio-tactile paradigms suggest that attentional orienting in time and modality are a closely coupled system, in which temporal orienting dominates modality orienting, similar to what happens in cross-modal spatial attention. On the other hand, recent findings using a visuo-tactile paradigm suggest that attentional orienting in time can unfold independently in each modality, leading to cross-modal decoupling. In the present study, we investigated if cross-modal decoupling in time can be extrapolated to audio-tactile contexts. If so, decoupling might represent a general property of cross-modal attention in time. To this end, we used a speeded discrimination task in which we manipulated the probability of target presentation in time and modality. In each trial, a manipulation of time-based expectancy was used to guide participants’ attention to task-relevant events, either tactile or auditory, at different points in time. In two experiments, we found that participants generally showed enhanced behavioural performance at the most likely onset time of each modality and no evidence for coupling. This pattern supports the hypothesis that cross-modal decoupling could be a general phenomenon in temporal orienting., This research was funded by the Spanish Ministry of Science and Innovation (PSI2016-75558-P(AEI/FEDER)), the Comissionat per a Universitats i Recerca del DIUE-Generalitat de Catalunya (2014SGR856), and the European Research Council (StG-2010 263145) to S.S.F.
Recercat. Dipósit de la Recerca de Catalunya
Publication Artículo científico (JournalArticle)

Beat gestures and syntactic parsing: an ERP study

  • Biau, Emmanuel, 1985-
  • Fromont, Lauren A.
  • Soto-Faraco, Salvador, 1970-
We tested the prosodic hypothesis that the temporal alignment of a speaker's beat gestures in a sentence influences syntactic parsing by driving the listener's attention. Participants chose between two possible interpretations of relative-clause (RC) ambiguous sentences, while their electroencephalogram (EEG) was recorded. We manipulated the alignment of the beat within sentences where auditory prosody was removed. Behavioral performance showed no effect of beat placement on the sentences’ interpretation, while event-related potentials (ERPs) revealed a positive shift of the signal in the windows corresponding to N100 and P200 components. Additionally, post hoc analyses of the ERPs time locked to the RC revealed a modulation of the P600 component as a function of gesture. These results suggest that beats modulate early processing of affiliate words in continuous speech and potentially have a global impact at the level of sentence-parsing components. We speculate that beats must be synergistic with auditory prosody to be fully consequential in behavior., This research was supported by the Ministerio de Econom´ıa y Competitividad (PSI2016-75558-P), AGAUR Generalitat de Catalunya (2014SGR856), and the European Research Council (StG-2010 263145). EB was supported by a postdoctoral fellowship from the European Union’s Horizon 2020 research and innovation programme, under the Marie Sklodowska-Curie grant agreement No. 707727
Recercat. Dipósit de la Recerca de Catalunya
Publication Artículo científico (JournalArticle)

Characteristic sounds facilitate object search in real-life scenes

  • Kvasova, Daria, 1989-
  • Garcia-Vernet, Laia
  • Soto-Faraco, Salvador, 1970-
Real-world events do not only provide temporally and spatially correlated information across the senses, but also semantic correspondences about object identity. Prior research has shown that object sounds can enhance detection, identification, and search performance of semantically consistent visual targets. However, these effects are always demonstrated in simple and stereotyped displays that lack ecological validity. In order to address identity-based cross-modal relationships in real-world scenarios, we designed a visual search task using complex, dynamic scenes. Participants searched for objects in video clips recorded from real-life scenes. Auditory cues, embedded in the background sounds, could be target-consistent, distracter-consistent, neutral, or just absent. We found that, in these naturalistic scenes, characteristic sounds improve visual search for task-relevant objects but fail to increase the salience of irrelevant distracters. Our findings generalize previous results on object-based cross-modal interactions with simple stimuli and shed light upon how audio–visual semantically congruent relationships play out in real-life contexts., This research was supported by the Ministerio de Economia y Competitividad (PSI2016-75558-P AEI/FEDER) and the AGAUR Generalitat de Catalunya (2017 SGR 1545). DK was supported by an FI scholarship, from the AGAUR Generalitat de Catalunya. This manuscript has been released as a pre-print at bioRxiv (Kvasova et al., 2019).
Dipòsit Digital de Documents de la UAB
Publication Artículo científico (JournalArticle)

Beliefs about others' intentions determine whether cooperation is the faster choice

  • Castro Santa, Juana|||0000-0002-6411-9460
  • Exadaktylos, Filippos|||0000-0001-7339-8847
  • Soto-Faraco, Salvador|||0000-0002-4799-3762
Is collaboration the fast choice for humans? Past studies proposed that cooperation is a behavioural default, based on Response Times (RT) findings. Here we contend that the individual's reckoning of the immediate social environment shapes her predisposition to cooperate and, hence, response latencies. In a social dilemma game, we manipulate the beliefs about the partner's intentions to cooperate and show that they act as a switch that determines cooperation and defection RTs; when the partner's intention to cooperate is perceived as high, cooperation choices are speeded up, while defection is slowed down. Importantly, this social context effect holds across varying expected payoffs, indicating that it modulates behaviour regardless of choices' similarity in monetary terms. Moreover, this pattern is moderated by individual variability in social preferences: Among conditional cooperators, high cooperation beliefs speed up cooperation responses and slow down defection. Among free-riders, defection is always faster and more likely than cooperation, while high cooperation beliefs slow down all decisions. These results shed new light on the conflict of choices account of response latencies, as well as on the intuitive cooperation hypothesis, and can help to correctly interpret and reconcile previous, apparently contradictory results, by considering the role of context in social dilemmas.