Visual speech improves the intelligibility of time-expanded auditory speech

Abstract

This study investigated the effects of intermodal timing differences and speed differences on word intelligibility of auditory–visual speech. Words were presented under visual-only, auditory-only, and auditory–visual conditions. Two types of auditory–visual conditions were used: asynchronous and expansion conditions. In the asynchronous conditions, the audio lag was 0–400 ms. In the expansion conditions, the auditory signal was time expanded (0–400 ms), whereas the visual signal was kept at the original speed. Results showed that word intelligibility was higher in the auditory–visual conditions than in the auditory-only condition. The results of auditory–visual benefit revealed that the benefit at the end of words declined as the amount of time expansion increased, although it did not decline in the asynchronous conditions.