Pasar al contenido principal

LA INTERACCION ENTRE INTEGRACION MULTISENSORIAL Y ATENCION

PSI2010-15426

•
Nombre agencia financiadora Ministerio de Ciencia e Innovación
Acrónimo agencia financiadora MICINN
Programa Programa Nacional de Proyectos de Investigación Fundamental
Subprograma Subprograma de Proyectos de Investigación Fundamental no Orientada
Convocatoria Proyectos I+D
Año convocatoria 2010
Plan Estatal 2008-2011 Plan Nacional de Investigación Científica, Desarrollo e Innovación Tecnológica
Unidad de gestión Subdirección General de Proyectos de Investigación
Centro beneficiario UNIVERSITAT POMPEU FABRA CCT
Identificador persistente http://dx.doi.org/10.13039/501100004837

Resultados relacionados

Resultados totales (Incluyendo duplicados): 22
Encontrada(s) 1 página(s)

Repositorio Digital de la UPF
Publicación Artículo científico (JournalArticle)

Effect of attentional load on audiovisual speech perception: evidence from ERPs

  • Alsius, Agnès
  • Möttönen, Riikka
  • Sams, Mikko E.
  • Soto-Faraco, Salvador, 1970-
  • Tiippana, Kaisa
Seeing articulatory movements influences perception of auditory speech. This is often reflected in a shortened latency of auditory event-related potentials (ERPs) generated in the auditory cortex. The present study addressed whether this early neural correlate of audiovisual interaction is modulated by attention. We recorded ERPs in 15 subjects while they were presented with auditory, visual, and audiovisual spoken syllables. Audiovisual stimuli consisted of incongruent auditory and visual components known to elicit a McGurk effect, i.e., a visually driven alteration in the auditory speech percept. In a Dual task condition, participants were asked to identify spoken syllables whilst monitoring a rapid visual stream of pictures for targets, i.e., they had to divide their attention. In a Single task condition, participants identified the syllables without any other tasks, i.e., they were asked to ignore the pictures and focus their attention fully on the spoken syllables. The McGurk effect was weaker in the Dual task than in the Single task condition, indicating an effect of attentional load on audiovisual speech perception. Early auditory ERP components, N1 and P2, peaked earlier to audiovisual stimuli than to auditory stimuli when attention was fully focused on syllables, indicating neurophysiological audiovisual interaction. This latency decrement was reduced when attention was loaded, suggesting that attention influences early neural processing of audiovisual speech. We conclude that reduced attention weakens the interaction between vision and audition in speech., This research was supported by grants from Spanish Ministry of Science and Innovation (PSI2010-15426) and the European Research Council (StG-2010 263145) to Salvador Soto-Faraco and Agnès Alsius.
Repositorio Digital de la UPF
Publicación Artículo científico (JournalArticle)

Age-related sensitive periods influence visual language discrimination in adults

  • Weikum, Whitney M.
  • Vouloumanos, Athena
  • Navarra, Jordi
  • Soto-Faraco, Salvador, 1970-
  • Sebastián Gallés, Núria
  • Werker, Janet F.
Adults as well as infants have the capacity to discriminate languages based on visual speech alone. Here, we investigated whether adults' ability to discriminate languages based on visual speech cues is influenced by the age of language acquisition. Adult participants who had all learned English (as a first or second language) but did not speak French were shown faces of bilingual (French/English) speakers silently reciting sentences in either language. Using only visual speech information, adults who had learned English from birth or as a second language before the age of 6 could discriminate between French and English significantly better than chance. However, adults who had learned English as a second language after age 6 failed to discriminate these two languages, suggesting that early childhood exposure is crucial for using relevant visual speech information to separate languages visually. These findings raise the possibility that lowered sensitivity to non-native visual speech cues may contribute to the difficulties encountered when learning a new language in adulthood., This research was supported by Research grants from the Natural Sciences and Engineering Research Council (NSERC) and the Social Sciences and Humanities Research Council (SSHRC) to Janet F. Werker, by NSERC and SSHRC Fellowships to Whitney M. Weikum, and by grants PSI2009-12859, PSI2012-39149 and RYC-2008-03672 from Ministerio de Economía y Competitividad (Spanish Government), and the European COST action TD0904 to Jordi Navarra. Salvador Soto-Faraco was supported by ERC (StG-2010263145), MICINN (PSI2010-15426 and Consolider INGENIO CSD2007-00012) and AGAUR (SGR2009-092).
Repositorio Digital de la UPF
Publicación Artículo científico (JournalArticle)

Alpha stimulation of the human parietal cortex attunes tactile perception to external space

  • Ruzzoli, Manuela
  • Soto-Faraco, Salvador, 1970-
An intriguing question in neuroscience concerns how somatosensory events on the skin are represented in the human brain. Since Head and Holmes’ [1] neuropsychological dissociation between localizing touch on the skin and localizing body parts in external space, touch is considered to operate in a variety of spatial reference frames [2]. At least two representations of space are in competition during orienting to touch: a somatotopic one, reflecting the organization of the somatosensory cortex (S1) [3], and a more abstract, external reference frame that factors postural changes in relation to body parts and/or external space [4, 5]. Previous transcranial magnetic stimulation (TMS) studies suggest that the posterior parietal cortex (PPC) plays a key role in supporting representations as well as orienting attention in an external reference frame [4, 6]. Here, we capitalized on the TMS entrainment approach [7, 8], targeting the intraparietal sulcus (IPS). We found that frequency-specific (10 Hz) tuning of the PPC induced spatially specific enhancement of tactile detection that was expressed in an external reference frame. This finding establishes a tight causal link between a concrete form of brain activity (10 Hz oscillation) and a specific type of spatial representation, revealing a fundamental property of how the parietal cortex encodes information., This work was supported by grants from the European Research Council (StG-2010 263145), the Spanish Ministry of Science and Innovation (PSI2010-15426), and the Comissionat per a Universitats i Recerca del DIUE-Generalitat de Catalunya (SRG2009-092) to S.S.-F.
Repositorio Digital de la UPF
Publicación Artículo científico (JournalArticle)

Visual information constrains early and late stages of spoken-word recognition in sentence context

  • Brunellière, Angèle
  • Sánchez García, Carolina, 1984-
  • Ikumi Montserrat, Nara, 1986-
  • Soto-Faraco, Salvador, 1970-
Audiovisual speech perception has been frequently studied considering phoneme, syllable and word processing levels. Here, we examined the constraints that visual speech information might exert during the recognition of words embedded in a natural sentence context. We recorded event-related potentials (ERPs) to words that could be either strongly or weakly predictable on the basis of the prior semantic sentential context and, whose initial phoneme varied in the degree of visual saliency from lip movements. When the sentences were presented audio-visually (Experiment 1), words weakly predicted from semantic context elicited a larger long-lasting N400, compared to strongly predictable words. This semantic effect interacted with the degree of visual saliency over a late part of the N400. When comparing audio-visual versus auditory alone presentation (Experiment 2), the typical amplitude-reduction effect over the auditory-evoked N100 response was observed in the audiovisual modality. Interestingly, a specific benefit of high- versus low-visual saliency constraints occurred over the early N100 response and at the late N400 time window, confirming the result of Experiment 1. Taken together, our results indicate that the saliency of visual speech can exert an influence over both auditory processing and word recognition at relatively late stages, and thus suggest strong interactivity between audio-visual integration and other (arguably higher) stages of information processing during natural speech comprehension., This research was supported by the Spanish Ministry of Science and Innovation (PSI2010-15426 and Consolider INGENIO CSD2007-00012), Comissionat per a Universitats i Recerca del DIUE-Generalitat de Catalunya (SGR2009-092), and the European Research Council (StG-2010263145).
Repositorio Digital de la UPF
Publicación Artículo científico (JournalArticle)

Discriminating speech rhythms in audition, vision, and touch

  • Navarra, Jordi
  • Soto-Faraco, Salvador, 1970-
  • Spence, Charles
We investigated the extent to which people can discriminate between languages on the basis of their characteristic temporal, rhythmic information, and the extent to which this ability generalizes across sensory modalities. We used rhythmical patterns derived from the alternation of vowels and consonants in English and Japanese, presented in audition, vision, both audition and vision at the same time, or touch. Experiment 1 confirmed that discrimination is possible on the basis of auditory rhythmic patterns, and extended it to the case of vision, using ‘aperture-close’ mouth movements of a schematic face. In Experiment 2, language discrimination was demonstrated using visual and auditory materials that did not resemble spoken articulation. In a combined analysis including data from Experiments 1 and 2, a beneficial effect was also found when the auditory rhythmic information was available to participants. Despite the fact that discrimination could be achieved using vision alone, auditory performance was nevertheless better. In a final experiment, we demonstrate that the rhythm of speech can also be discriminated successfully by means of vibrotactile patterns delivered to the fingertip. The results of the present study therefore demonstrate that discrimination between language's syllabic rhythmic patterns is possible on the basis of visual and tactile displays., This research was supported by grants PSI2012-39149, PSI2009-12859 and RYC-2008-03672 from Ministerio de Economía y Competitividad (MINECO, Spain) to J. Navarra, the European COST action TD0904, and by grants PSI2010-15426 and CDS00012 from MINECO (Spain), 2009SGR-292 from Comissionat per a Universitats i Recerca del DIUE-Generalitat de Catalunya, and European Research Council (StG-2010 263145) to S. Soto-Faraco.
Repositorio Digital de la UPF
Publicación Artículo científico (JournalArticle)

Isolating shape from semantics in haptic-visual priming

  • Pesquita, Ana
  • Brennan, Allison A.
  • Enns, James T.
  • Soto-Faraco, Salvador, 1970-
The exploration of a familiar object by hand can benefit its identification by eye. What is unclear is how much this multisensory cross-talk reflects shared shape representations versus generic semantic associations. Here, we compare several simultaneous priming conditions to isolate the potential contributions of shape and semantics in haptic-to-visual priming. Participants explored a familiar object manually (haptic prime) while trying to name a visual object that was gradually revealed in increments of spatial resolution. Shape priming was isolated in a comparison of identity priming (shared semantic category and shape) with category priming (same category, but different shapes). Semantic priming was indexed by the comparisons of category priming with unrelated haptic primes. The results showed that both factors mediated priming, but that their relative weights depended on the reliability of the visual information. Semantic priming dominated in Experiment 1, when participants were free to use high-resolution visual information, but shape priming played a stronger role in Experiment 2, when participants were forced to respond with less reliable visual information. These results support the structural description hypothesis of haptic-visual priming (Reales and Ballesteros in J Exp Psychol Learn Mem Cogn 25:644–663, 1999) and are also consistent with the optimal integration theory (Ernst and Banks in Nature 415:429–433, 2002), which proposes a close coupling between the reliability of sensory signals and their weight in decision making., This work was supported by a PhD scholarship to author AP from the Portuguese Fundação para a Ciência e Tecnologia (SFRH/BD/76087/2011), an Exchange Fellowship to author AAB from the Dr. Michael Quinn Memorial Fund, Department of Psychology, University of British Columbia, an NSERC (Canada) Discovery Grant to author JTE, and Grants to author SS-F from the Spanish Ministry of Science and Innovation (PSI2010-15426 and Consolider INGENIO CSD2007-00012 Grants, the Comissionat per a Universitats i Recerca del DIUE-Generalitat de Catalunya (SRG2009- 092), and the European Research Council (StG-2010 263145).
Repositorio Digital de la UPF
Publicación Artículo científico (JournalArticle)

Electrophysiological correlates of tactile remapping

  • Soto-Faraco, Salvador, 1970-
  • Azañón Gracia, Elena
Orienting our gaze or attention to the location of tactile events in the skin feels natural and effortless. However, this process requires combining somatosensory and proprioceptive information in a non-trivial, time consuming, fashion. Here we address the time course of tactile remapping, from somatotopically-based representations, to a spatiotopic reference frame. We compared electrical responses at the scalp evoked by touch at one finger as participants held their arms straight or else, crossed about the body midline. This postural manipulation creates a conflict between frames of reference, which can be used to reveal the consequences of spatial remapping. Behavioural performance was gauged online for crossed and uncrossed postures using bimanual temporal order judgment (TOJ) trials occurring occasionally during the recording session. The first electrophysiological signs of tactile remapping were observed around 70 ms after the tactile event, overlapping in time with the somatosensory component N80. This electrophysiological effect was strongly lateralized to the left scalp and independent of the hand being stimulated. Moreover, participants who manifested a stronger behavioural conflict between frames of reference in the TOJ task displayed a larger electrophysiological effect. Based on these findings, and the known properties of the somatosensory network, we argue that remapping of tactile space depends on fast feedback projections from association areas of the parietal cortex, and encompasses a left-lateralized fronto-parietal network supporting the selection of guided actions., This research was supported by the Spanish Ministry of Science and Innovation (PSI2010-15426 and Consolider INGENIO CSD2007-00012), Comissionat per a Universitats i Recerca del DIUE-Generalitat de Catalunya (SRG2009-092), and the European Research Council (StG-2010 263145).
Repositorio Digital de la UPF
Publicación Artículo científico (JournalArticle)

Acoustic facilitation of object movement detection during self-motion

  • Calabro, Finnegan
  • Soto-Faraco, Salvador, 1970-
  • Vaina, Lucia M.
In humans, as well as most animal species, perception of object motion is critical to successful interaction with the surrounding environment. Yet, as the observer also moves, the retinal projections of the various motion components add to each other and extracting accurate object motion becomes computationally challenging. Recent psychophysical studies have demonstrated that observers use a flow-parsing mechanism to estimate and subtract self-motion from the optic flow field. We investigated whether concurrent acoustic cues for motion can facilitate visual flow parsing, thereby enhancing the detection of moving objects during simulated self-motion. Participants identified an object (the target) that moved either forward or backward within a visual scene containing nine identical textured objects simulating forward observer translation. We found that spatially co-localized, directionally congruent, moving auditory stimuli enhanced object motion detection. Interestingly, subjects who performed poorly on the visual-only task benefited more from the addition of moving auditory stimuli. When auditory stimuli were not co-localized to the visual target, improvements in detection rates were weak. Taken together, these results suggest that parsing object motion from self-motion-induced optic flow can operate on multisensory object representations., L.M.V. and F.J.C. were supported by NIH grant RO1NS064100 to L.M.V. S.S.F. was supported by grants from the Ministerio de Ciencia e Innovación (PSI2010-15426 and CSD2007-00012), by the Comissionat per a Universitats i Recerca del DIUE (SRG2009-092) and by the European Research Council (StG-2010 263145).
Repositorio Digital de la UPF
Publicación Artículo científico (JournalArticle)

The Interplay between semantic and phonological constraints during spoken-word comprehension

  • Soto-Faraco, Salvador, 1970-
  • Brunellière, Angèle
This study addresses how top-down predictions driven by phonological and semantic information interact on spokenword comprehension. To do so, we measured event-related potentials to words embedded in sentences that varied in the degree of semantic constraint (high or low) and in regional accent (congruent or incongruent) with respect to the target word pronunciation. The data showed a negative amplitude shift following phonological mismatch (target pronunciation incongruent with respect to sentence regional accent). Here, we show that this shift is modulated by sentence-level semantic constraints over latencies encompassing auditory (N100) and lexical (N400) components. These findings suggest a fast influence of top-down predictions and the interplay with bottom-up processes at sublexical and lexical levels of analysis, This research was supported by the Spanish Ministry of Science and Innovation (PSI2010-15426 and Consolider INGENIO CSD2007-00012), Comissionat per a Universitats i Recerca del DIUE-Generalitat de/nCatalunya (SGR2009-092), and the European Research Council (StG-2010263145). ERP analyses were performed with the Cartool software (supported by the Center for Biomedical Imaging of Geneva and Lausanne).
Repositorio Digital de la UPF
Publicación Artículo científico (JournalArticle)

Beat gestures modulate auditory integration in speech perception

  • Biau, Emmanuel, 1985-
  • Soto-Faraco, Salvador, 1970-
Spontaneous beat gestures are an integral part of the paralinguistic context during face-to-face conversations. Here we investigated the time course of beat-speech integration in speech perception by measuring ERPs evoked by words pronounced with or without an accompanying beat gesture, while participants watched a spoken discourse. Words accompanied by beats elicited a positive shift in ERPs at an early sensory stage (before 100 ms) and at a later time window coinciding with the auditory component P2. The same word tokens produced no ERP differences when participants listened to the discourse without view of the speaker. We conclude that beat gestures are integrated with speech early on in time and modulate sensory/phonological levels of processing. The present results support the possible role of beats as a highlighter, helping the listener to direct the focus of attention to important information and modulate the parsing of the speech stream., This research was supported by the Spanish Ministry of Science and Innovation (PSI2010-15426 and Consolider INGENIO CSD2007-00012), Comissionat per a Universitats i Recerca del DIUE-Generalitat de Catalunya (SRG2009-092), and the European Research Council (StG-2010 263145).
Repositorio Digital de la UPF
Publicación Artículo científico (JournalArticle)

The speakers’ accent shapes the listeners’ phonological predictions during speech perception

  • Brunellière, Angèle
  • Soto-Faraco, Salvador, 1970-
This study investigates the specificity of predictive coding in spoken word comprehension using event-related potentials (ERPs). We measured word-evoked ERPs in Catalan speakers listening to semantically constraining sentences produced in their native regional accent (Experiment 1) or in a non-native accent (Experiment 2). Semantically anomalous words produced long-lasting negative shift (N400) starting as early as 250 ms, thus reflecting phonological as well as semantic mismatch. Semantically expected but phonologically unexpected (non-native forms embedded in a native context) produced only an early (∼250 ms) negative difference. In contrast, this phonological expectancy effect failed for native albeit phonologically unexpected target words embedded in a non-native context. These results suggest phonologically precise expectations when operating over native input, whilst phonologically less specified expectations in a non-native context. Our findings shed light on contextual influence during word recognition, suggesting that word form prediction based on context is sensitive and adaptive to phonological variability., his research was supported by the Spanish Ministry of Science and Innovation (PSI2010-15426 and Consolider INGENIO CSD2007-00012), Comissionat per a Universitats i Recerca del DIUE-Generalitat de Catalunya (SGR2009-092), and the European Research Council (StG-2010263145).
Recercat. Dipósit de la Recerca de Catalunya
Publicación Artículo científico (JournalArticle)

The speakers’ accent shapes the listeners’ phonological predictions during speech perception

  • Brunellière, Angèle
  • Soto-Faraco, Salvador, 1970-
This study investigates the specificity of predictive coding in spoken word comprehension using event-related potentials (ERPs). We measured word-evoked ERPs in Catalan speakers listening to semantically constraining sentences produced in their native regional accent (Experiment 1) or in a non-native accent (Experiment 2). Semantically anomalous words produced long-lasting negative shift (N400) starting as early as 250 ms, thus reflecting phonological as well as semantic mismatch. Semantically expected but phonologically unexpected (non-native forms embedded in a native context) produced only an early (∼250 ms) negative difference. In contrast, this phonological expectancy effect failed for native albeit phonologically unexpected target words embedded in a non-native context. These results suggest phonologically precise expectations when operating over native input, whilst phonologically less specified expectations in a non-native context. Our findings shed light on contextual influence during word recognition, suggesting that word form prediction based on context is sensitive and adaptive to phonological variability., his research was supported by the Spanish Ministry of Science and Innovation (PSI2010-15426 and Consolider INGENIO CSD2007-00012), Comissionat per a Universitats i Recerca del DIUE-Generalitat de Catalunya (SGR2009-092), and the European Research Council (StG-2010263145).
Recercat. Dipósit de la Recerca de Catalunya
Publicación Artículo científico (JournalArticle)

Isolating shape from semantics in haptic-visual priming

  • Pesquita, Ana
  • Brennan, Allison A.
  • Enns, James T.
  • Soto-Faraco, Salvador, 1970-
The exploration of a familiar object by hand can benefit its identification by eye. What is unclear is how much this multisensory cross-talk reflects shared shape representations versus generic semantic associations. Here, we compare several simultaneous priming conditions to isolate the potential contributions of shape and semantics in haptic-to-visual priming. Participants explored a familiar object manually (haptic prime) while trying to name a visual object that was gradually revealed in increments of spatial resolution. Shape priming was isolated in a comparison of identity priming (shared semantic category and shape) with category priming (same category, but different shapes). Semantic priming was indexed by the comparisons of category priming with unrelated haptic primes. The results showed that both factors mediated priming, but that their relative weights depended on the reliability of the visual information. Semantic priming dominated in Experiment 1, when participants were free to use high-resolution visual information, but shape priming played a stronger role in Experiment 2, when participants were forced to respond with less reliable visual information. These results support the structural description hypothesis of haptic-visual priming (Reales and Ballesteros in J Exp Psychol Learn Mem Cogn 25:644–663, 1999) and are also consistent with the optimal integration theory (Ernst and Banks in Nature 415:429–433, 2002), which proposes a close coupling between the reliability of sensory signals and their weight in decision making., This work was supported by a PhD scholarship to author AP from the Portuguese Fundação para a Ciência e Tecnologia (SFRH/BD/76087/2011), an Exchange Fellowship to author AAB from the Dr. Michael Quinn Memorial Fund, Department of Psychology, University of British Columbia, an NSERC (Canada) Discovery Grant to author JTE, and Grants to author SS-F from the Spanish Ministry of Science and Innovation (PSI2010-15426 and Consolider INGENIO CSD2007-00012 Grants, the Comissionat per a Universitats i Recerca del DIUE-Generalitat de Catalunya (SRG2009- 092), and the European Research Council (StG-2010 263145).
Recercat. Dipósit de la Recerca de Catalunya
Publicación Artículo científico (JournalArticle)

Discriminating speech rhythms in audition, vision, and touch

  • Navarra, Jordi
  • Soto-Faraco, Salvador, 1970-
  • Spence, Charles
We investigated the extent to which people can discriminate between languages on the basis of their characteristic temporal, rhythmic information, and the extent to which this ability generalizes across sensory modalities. We used rhythmical patterns derived from the alternation of vowels and consonants in English and Japanese, presented in audition, vision, both audition and vision at the same time, or touch. Experiment 1 confirmed that discrimination is possible on the basis of auditory rhythmic patterns, and extended it to the case of vision, using ‘aperture-close’ mouth movements of a schematic face. In Experiment 2, language discrimination was demonstrated using visual and auditory materials that did not resemble spoken articulation. In a combined analysis including data from Experiments 1 and 2, a beneficial effect was also found when the auditory rhythmic information was available to participants. Despite the fact that discrimination could be achieved using vision alone, auditory performance was nevertheless better. In a final experiment, we demonstrate that the rhythm of speech can also be discriminated successfully by means of vibrotactile patterns delivered to the fingertip. The results of the present study therefore demonstrate that discrimination between language's syllabic rhythmic patterns is possible on the basis of visual and tactile displays., This research was supported by grants PSI2012-39149, PSI2009-12859 and RYC-2008-03672 from Ministerio de Economía y Competitividad (MINECO, Spain) to J. Navarra, the European COST action TD0904, and by grants PSI2010-15426 and CDS00012 from MINECO (Spain), 2009SGR-292 from Comissionat per a Universitats i Recerca del DIUE-Generalitat de Catalunya, and European Research Council (StG-2010 263145) to S. Soto-Faraco.
Recercat. Dipósit de la Recerca de Catalunya
Publicación Artículo científico (JournalArticle)

Beat gestures modulate auditory integration in speech perception

  • Biau, Emmanuel, 1985-
  • Soto-Faraco, Salvador, 1970-
Spontaneous beat gestures are an integral part of the paralinguistic context during face-to-face conversations. Here we investigated the time course of beat-speech integration in speech perception by measuring ERPs evoked by words pronounced with or without an accompanying beat gesture, while participants watched a spoken discourse. Words accompanied by beats elicited a positive shift in ERPs at an early sensory stage (before 100 ms) and at a later time window coinciding with the auditory component P2. The same word tokens produced no ERP differences when participants listened to the discourse without view of the speaker. We conclude that beat gestures are integrated with speech early on in time and modulate sensory/phonological levels of processing. The present results support the possible role of beats as a highlighter, helping the listener to direct the focus of attention to important information and modulate the parsing of the speech stream., This research was supported by the Spanish Ministry of Science and Innovation (PSI2010-15426 and Consolider INGENIO CSD2007-00012), Comissionat per a Universitats i Recerca del DIUE-Generalitat de Catalunya (SRG2009-092), and the European Research Council (StG-2010 263145).
Recercat. Dipósit de la Recerca de Catalunya
Publicación Artículo científico (JournalArticle)

Acoustic facilitation of object movement detection during self-motion

  • Calabro, Finnegan
  • Soto-Faraco, Salvador, 1970-
  • Vaina, Lucia M.
In humans, as well as most animal species, perception of object motion is critical to successful interaction with the surrounding environment. Yet, as the observer also moves, the retinal projections of the various motion components add to each other and extracting accurate object motion becomes computationally challenging. Recent psychophysical studies have demonstrated that observers use a flow-parsing mechanism to estimate and subtract self-motion from the optic flow field. We investigated whether concurrent acoustic cues for motion can facilitate visual flow parsing, thereby enhancing the detection of moving objects during simulated self-motion. Participants identified an object (the target) that moved either forward or backward within a visual scene containing nine identical textured objects simulating forward observer translation. We found that spatially co-localized, directionally congruent, moving auditory stimuli enhanced object motion detection. Interestingly, subjects who performed poorly on the visual-only task benefited more from the addition of moving auditory stimuli. When auditory stimuli were not co-localized to the visual target, improvements in detection rates were weak. Taken together, these results suggest that parsing object motion from self-motion-induced optic flow can operate on multisensory object representations., L.M.V. and F.J.C. were supported by NIH grant RO1NS064100 to L.M.V. S.S.F. was supported by grants from the Ministerio de Ciencia e Innovación (PSI2010-15426 and CSD2007-00012), by the Comissionat per a Universitats i Recerca del DIUE (SRG2009-092) and by the European Research Council (StG-2010 263145).
Recercat. Dipósit de la Recerca de Catalunya
Publicación Artículo científico (JournalArticle)

Visual information constrains early and late stages of spoken-word recognition in sentence context

  • Brunellière, Angèle
  • Sánchez García, Carolina, 1984-
  • Ikumi Montserrat, Nara, 1986-
  • Soto-Faraco, Salvador, 1970-
Audiovisual speech perception has been frequently studied considering phoneme, syllable and word processing levels. Here, we examined the constraints that visual speech information might exert during the recognition of words embedded in a natural sentence context. We recorded event-related potentials (ERPs) to words that could be either strongly or weakly predictable on the basis of the prior semantic sentential context and, whose initial phoneme varied in the degree of visual saliency from lip movements. When the sentences were presented audio-visually (Experiment 1), words weakly predicted from semantic context elicited a larger long-lasting N400, compared to strongly predictable words. This semantic effect interacted with the degree of visual saliency over a late part of the N400. When comparing audio-visual versus auditory alone presentation (Experiment 2), the typical amplitude-reduction effect over the auditory-evoked N100 response was observed in the audiovisual modality. Interestingly, a specific benefit of high- versus low-visual saliency constraints occurred over the early N100 response and at the late N400 time window, confirming the result of Experiment 1. Taken together, our results indicate that the saliency of visual speech can exert an influence over both auditory processing and word recognition at relatively late stages, and thus suggest strong interactivity between audio-visual integration and other (arguably higher) stages of information processing during natural speech comprehension., This research was supported by the Spanish Ministry of Science and Innovation (PSI2010-15426 and Consolider INGENIO CSD2007-00012), Comissionat per a Universitats i Recerca del DIUE-Generalitat de Catalunya (SGR2009-092), and the European Research Council (StG-2010263145).
Recercat. Dipósit de la Recerca de Catalunya
Publicación Artículo científico (JournalArticle)

Age-related sensitive periods influence visual language discrimination in adults

  • Weikum, Whitney M.
  • Vouloumanos, Athena
  • Navarra, Jordi
  • Soto-Faraco, Salvador, 1970-
  • Sebastián Gallés, Núria
  • Werker, Janet F.
Adults as well as infants have the capacity to discriminate languages based on visual speech alone. Here, we investigated whether adults' ability to discriminate languages based on visual speech cues is influenced by the age of language acquisition. Adult participants who had all learned English (as a first or second language) but did not speak French were shown faces of bilingual (French/English) speakers silently reciting sentences in either language. Using only visual speech information, adults who had learned English from birth or as a second language before the age of 6 could discriminate between French and English significantly better than chance. However, adults who had learned English as a second language after age 6 failed to discriminate these two languages, suggesting that early childhood exposure is crucial for using relevant visual speech information to separate languages visually. These findings raise the possibility that lowered sensitivity to non-native visual speech cues may contribute to the difficulties encountered when learning a new language in adulthood., This research was supported by Research grants from the Natural Sciences and Engineering Research Council (NSERC) and the Social Sciences and Humanities Research Council (SSHRC) to Janet F. Werker, by NSERC and SSHRC Fellowships to Whitney M. Weikum, and by grants PSI2009-12859, PSI2012-39149 and RYC-2008-03672 from Ministerio de Economía y Competitividad (Spanish Government), and the European COST action TD0904 to Jordi Navarra. Salvador Soto-Faraco was supported by ERC (StG-2010263145), MICINN (PSI2010-15426 and Consolider INGENIO CSD2007-00012) and AGAUR (SGR2009-092).
Recercat. Dipósit de la Recerca de Catalunya
Publicación Artículo científico (JournalArticle)

The Interplay between semantic and phonological constraints during spoken-word comprehension

  • Soto-Faraco, Salvador, 1970-
  • Brunellière, Angèle
This study addresses how top-down predictions driven by phonological and semantic information interact on spokenword comprehension. To do so, we measured event-related potentials to words embedded in sentences that varied in the degree of semantic constraint (high or low) and in regional accent (congruent or incongruent) with respect to the target word pronunciation. The data showed a negative amplitude shift following phonological mismatch (target pronunciation incongruent with respect to sentence regional accent). Here, we show that this shift is modulated by sentence-level semantic constraints over latencies encompassing auditory (N100) and lexical (N400) components. These findings suggest a fast influence of top-down predictions and the interplay with bottom-up processes at sublexical and lexical levels of analysis, This research was supported by the Spanish Ministry of Science and Innovation (PSI2010-15426 and Consolider INGENIO CSD2007-00012), Comissionat per a Universitats i Recerca del DIUE-Generalitat de/nCatalunya (SGR2009-092), and the European Research Council (StG-2010263145). ERP analyses were performed with the Cartool software (supported by the Center for Biomedical Imaging of Geneva and Lausanne).
Recercat. Dipósit de la Recerca de Catalunya
Publicación Artículo científico (JournalArticle)

Alpha stimulation of the human parietal cortex attunes tactile perception to external space

  • Ruzzoli, Manuela
  • Soto-Faraco, Salvador, 1970-
An intriguing question in neuroscience concerns how somatosensory events on the skin are represented in the human brain. Since Head and Holmes’ [1] neuropsychological dissociation between localizing touch on the skin and localizing body parts in external space, touch is considered to operate in a variety of spatial reference frames [2]. At least two representations of space are in competition during orienting to touch: a somatotopic one, reflecting the organization of the somatosensory cortex (S1) [3], and a more abstract, external reference frame that factors postural changes in relation to body parts and/or external space [4, 5]. Previous transcranial magnetic stimulation (TMS) studies suggest that the posterior parietal cortex (PPC) plays a key role in supporting representations as well as orienting attention in an external reference frame [4, 6]. Here, we capitalized on the TMS entrainment approach [7, 8], targeting the intraparietal sulcus (IPS). We found that frequency-specific (10 Hz) tuning of the PPC induced spatially specific enhancement of tactile detection that was expressed in an external reference frame. This finding establishes a tight causal link between a concrete form of brain activity (10 Hz oscillation) and a specific type of spatial representation, revealing a fundamental property of how the parietal cortex encodes information., This work was supported by grants from the European Research Council (StG-2010 263145), the Spanish Ministry of Science and Innovation (PSI2010-15426), and the Comissionat per a Universitats i Recerca del DIUE-Generalitat de Catalunya (SRG2009-092) to S.S.-F.
Recercat. Dipósit de la Recerca de Catalunya
Publicación Artículo científico (JournalArticle)

Effect of attentional load on audiovisual speech perception: evidence from ERPs

  • Alsius, Agnès
  • Möttönen, Riikka
  • Sams, Mikko E.
  • Soto-Faraco, Salvador, 1970-
  • Tiippana, Kaisa
Seeing articulatory movements influences perception of auditory speech. This is often reflected in a shortened latency of auditory event-related potentials (ERPs) generated in the auditory cortex. The present study addressed whether this early neural correlate of audiovisual interaction is modulated by attention. We recorded ERPs in 15 subjects while they were presented with auditory, visual, and audiovisual spoken syllables. Audiovisual stimuli consisted of incongruent auditory and visual components known to elicit a McGurk effect, i.e., a visually driven alteration in the auditory speech percept. In a Dual task condition, participants were asked to identify spoken syllables whilst monitoring a rapid visual stream of pictures for targets, i.e., they had to divide their attention. In a Single task condition, participants identified the syllables without any other tasks, i.e., they were asked to ignore the pictures and focus their attention fully on the spoken syllables. The McGurk effect was weaker in the Dual task than in the Single task condition, indicating an effect of attentional load on audiovisual speech perception. Early auditory ERP components, N1 and P2, peaked earlier to audiovisual stimuli than to auditory stimuli when attention was fully focused on syllables, indicating neurophysiological audiovisual interaction. This latency decrement was reduced when attention was loaded, suggesting that attention influences early neural processing of audiovisual speech. We conclude that reduced attention weakens the interaction between vision and audition in speech., This research was supported by grants from Spanish Ministry of Science and Innovation (PSI2010-15426) and the European Research Council (StG-2010 263145) to Salvador Soto-Faraco and Agnès Alsius.
Recercat. Dipósit de la Recerca de Catalunya
Publicación Artículo científico (JournalArticle)

Electrophysiological correlates of tactile remapping

  • Soto-Faraco, Salvador, 1970-
  • Azañón Gracia, Elena
Orienting our gaze or attention to the location of tactile events in the skin feels natural and effortless. However, this process requires combining somatosensory and proprioceptive information in a non-trivial, time consuming, fashion. Here we address the time course of tactile remapping, from somatotopically-based representations, to a spatiotopic reference frame. We compared electrical responses at the scalp evoked by touch at one finger as participants held their arms straight or else, crossed about the body midline. This postural manipulation creates a conflict between frames of reference, which can be used to reveal the consequences of spatial remapping. Behavioural performance was gauged online for crossed and uncrossed postures using bimanual temporal order judgment (TOJ) trials occurring occasionally during the recording session. The first electrophysiological signs of tactile remapping were observed around 70 ms after the tactile event, overlapping in time with the somatosensory component N80. This electrophysiological effect was strongly lateralized to the left scalp and independent of the hand being stimulated. Moreover, participants who manifested a stronger behavioural conflict between frames of reference in the TOJ task displayed a larger electrophysiological effect. Based on these findings, and the known properties of the somatosensory network, we argue that remapping of tactile space depends on fast feedback projections from association areas of the parietal cortex, and encompasses a left-lateralized fronto-parietal network supporting the selection of guided actions., This research was supported by the Spanish Ministry of Science and Innovation (PSI2010-15426 and Consolider INGENIO CSD2007-00012), Comissionat per a Universitats i Recerca del DIUE-Generalitat de Catalunya (SRG2009-092), and the European Research Council (StG-2010 263145).