Sometimes information from our senses is merged so that what we experience reflects a sensory combination, quite different from what any one sense would have told us on its own. Nowhere is this more striking than in the McGurk Effect. In this illusion, the sound of a person saying one thing (e.g. the sound “BA”) is played over a video showing their lips saying something else (e.g. “GA”) , with the result that you hear them saying a mixture of the two (e.g. “DA”). First watch this video: http://tinyurl.com/4oj73 (QuickTime required: http://tinyurl.com/d3w3t) and then close your eyes and listen to it again. Notice the difference? The McGurk Effect tricks your brain’s multi-sensory strategy that in other circumstances – such as trying to listen to your friend at a noisy nightclub – helps you interpret what they’re saying.
Until now this combining of the senses was thought to occur automatically and without the need for any extra attention. But now a study by researchers at Barcelona University shows that if a person’s attentional resources are deployed on some other task, then visual-auditory integration won’t occur and they won’t experience the McGurk Effect, or at least not as much.
They played student volunteers a series of McGurk Effect stimuli. Half the students just had to report what they thought the talking face had said. The other half, however, had to complete a second task at the same time. This involved either looking at line drawings that were superimposed over the talking face and noting any repeats, or listening to sound effects and noting any repeats.
The researchers found that the McGurk Effect was reduced substantially in those students who had to complete secondary visual (Effect occurred 24 per cent less) or auditory (25 per cent less) tasks at the same time as they observed the McGurk stimuli. That is, when a participant was concentrating on another task at the same time, the actual word spoken by the face was perceived accurately, rather than being merged with the face’s lip movements.
“Exhausting attentional resources seriously compromises the multi-sensory integration process”, the authors said.
Alsuius, A. Navarra, J., Campbell, R. & Soto-Faraco, S. (2005). Audiovisual integration of speech falters under high attention demands. Current Biology, 15, 839-843.