Skip to main content
Accessibility|Contact|Privacy|Terms of Service

Multiple Senses Used in Speech Perception

  • Published: 2009-02-11 (Revised/Updated 2014-01-23) : Author: Association for Psychological Science
  • Synopsis: Recent studies suggest that we use a variety of senses for speech perception including lip reading.

Main Document

"Rosenblum suggests that physical movement of speech (that is, our mouths and lips moving) create acoustic and visual signals which have a similar form."

When someone speaks to you, do you see what they are saying? We tend to think of speech as being something we hear, but recent studies suggest that we use a variety of senses for speech perception - that the brain treats speech as something we hear, see and even feel.

In a new report in Current Directions in Psychological Science, a journal of the Association for Psychological Science, psychologist Lawrence Rosenblum describes research examining how our different senses blend together to help us perceive speech.

We receive a lot of our speech information via visual cues, such as lip-reading, and this type of visual speech occurs throughout all cultures. And it is not just information from lips- when someone is speaking to us, we will also note movements of the teeth, tongue and other non-mouth facial features. It's likely that human speech perception has evolved to integrate many senses together. Put in another way, speech is not meant to be just heard, but also to be seen.

The McGurk Effect is a well-characterized example of the integration between what we see and what we hear when someone is speaking to us.

This phenomenon occurs when a sound (such as a syllable or word) is dubbed with a video showing a face making a different sound. For example, the audio may be playing "ba," while the face looks as though it is saying "va." When confronted with this, we will usually hear "va" or a combination of the two sounds, such as "da." Interestingly, when study participants are aware of the dubbing or told to concentrate only on the audio, the McGurk Effect still occurs. Rosenblum suggests that this is evidence that once senses are integrated together, it is not possible to separate them.

Recent studies indicate that this integration occurs very early in the speech process, even before phonemes (the basic units of speech) are established.

Rosenblum suggests that physical movement of speech (that is, our mouths and lips moving) create acoustic and visual signals which have a similar form. He argues that as far as the speech brain is concerned, the auditory and visual information are never really separate. This could explain why we integrate speech so readily and in such a way that the audio and visual speech signals become indistinguishable from one another.

Rosenblum concludes that visual-speech research has a number of clinical implications, especially in the areas of autism, brain injury and schizophrenia and that "rehabilitation programs in each of these domains have incorporated visual-speech stimuli."

Similar Topics

1 : Using Sign Language Builds Phrases with Similar Neural Mechanisms as Speaking : New York University.
2 : Hearing Charities of America's Hearing Aid Project Changing Lives with Donor's Help : Hearing Charities of America.
3 : Brain Imaging Predicts Language Learning in Deaf Children : Ann & Robert H. Lurie Children's Hospital of Chicago.
4 : Sign Language Comparative List of Astronomical Words : International Astronomical Union.
5 : Sign Language May Offer Answer to Meaning of Music : New York University.
From our Deaf Communication section - Full List (64 Items)


Submit disability news, coming events, as well as assistive technology product news and reviews.


Loan Information for low income singles, families, seniors and disabled. Includes home, vehicle and personal loans.


Famous People with Disabilities - Well known people with disabilities and conditions who contributed to society.


List of awareness ribbon colors and their meaning. Also see our calendar of awareness dates.


Blood Pressure Chart - What should your blood pressure be, and information on blood group types/compatibility.





1 : Millennials Fail to Understand Dangers of Tanning
2 : Appetite Loss After Exercising Explained
3 : Bias Keeps Women with Higher Body Weight Away From the Doctor
4 : Smart Hoteliers are Building a Healthier Future
5 : Teaching Baby Sign Language - Nita, Show Us More
6 : MitoQ Novel Antioxidant Makes Old Arteries Seem Young Again
7 : Telemedicine Helps Overcome Healthcare Gender Based Barriers
8 : Screen Reader Plus Keyboard Helps Blind, Low-Vision Users Browse Modern Webpages


Disclaimer: This site does not employ and is not overseen by medical professionals. Content on Disabled World is not intended to be a substitute for professional medical advice, diagnosis, or treatment. Always seek the advice of a physician or other qualified health provider with any questions you may have regarding a medical condition. See our Terms of Service for more information.

Reporting Errors: Disabled World is an independent website, your assistance in reporting outdated or inaccurate information is appreciated. If you find an error please let us know.

© 2004 - 2018 Disabled World™