Secondary Logo

Journal Logo

Institutional members access full text with Ovid®

Neural processes underlying perceptual enhancement by visual speech gestures

Callan, Daniel E.1 CA; Jones, Jeffery A.1 2; Munhall, Kevin1 3; Callan, Akiko M.4; Kroos, Christian5; Vatikiotis-Bateson, Eric1 6


This fMRI study explores brain regions involved with perceptual enhancement afforded by observation of visual speech gesture information. Subjects passively identified words presented in the following conditions: audio-only, audiovisual, audio-only with noise, audiovisual with noise, and visual only. The brain may use concordant audio and visual information to enhance perception by integrating the information in a converging multisensory site. Consistent with response properties of multisensory integration sites, enhanced activity in middle and superior temporal gyrus/sulcus was greatest when concordant audiovisual stimuli were presented with acoustic noise. Activity found in brain regions involved with planning and execution of speech production in response to visual speech presented with degraded or absent auditory stimulation, is consistent with the use of an additional pathway through which speech perception is facilitated by a process of internally simulating the intended speech act of the observed speaker.

1Human Information Science Laboratories and

4Brain Activity Imaging Center, ATR International, 2-2-2 Hikaridai, Seika-cho, Soraku-gun, Kyoto 619-0288, Japan

2Department of Psychology, Wilfrid Laurier University

3Department of Psychology, Queens University, Canada

5Institut fuer Phonetik und Sprachliche Kommunication, Munich University, Germany

6Department of Linguistics, University of British Columbia, Canada

CACorresponding Author:

Received 25 July 2003; accepted 14 August 2003

© 2003 Lippincott Williams & Wilkins, Inc.