Skip to main content
Accessibility  |  About  |  Contact  |  Privacy  |  Terms

Multiple Senses Used in Speech Perception

  • Published: 2009-02-11 (Revised/Updated 2014-01-23) : Association for Psychological Science.
  • Synopsis: Recent studies suggest that we use a variety of senses for speech perception including lip reading.

Main Document

Quote: "Rosenblum suggests that physical movement of speech (that is, our mouths and lips moving) create acoustic and visual signals which have a similar form."

When someone speaks to you, do you see what they are saying? We tend to think of speech as being something we hear, but recent studies suggest that we use a variety of senses for speech perception - that the brain treats speech as something we hear, see and even feel.

In a new report in Current Directions in Psychological Science, a journal of the Association for Psychological Science, psychologist Lawrence Rosenblum describes research examining how our different senses blend together to help us perceive speech.

We receive a lot of our speech information via visual cues, such as lip-reading, and this type of visual speech occurs throughout all cultures. And it is not just information from lips- when someone is speaking to us, we will also note movements of the teeth, tongue and other non-mouth facial features. It's likely that human speech perception has evolved to integrate many senses together. Put in another way, speech is not meant to be just heard, but also to be seen.

The McGurk Effect is a well-characterized example of the integration between what we see and what we hear when someone is speaking to us.

This phenomenon occurs when a sound (such as a syllable or word) is dubbed with a video showing a face making a different sound. For example, the audio may be playing "ba," while the face looks as though it is saying "va." When confronted with this, we will usually hear "va" or a combination of the two sounds, such as "da." Interestingly, when study participants are aware of the dubbing or told to concentrate only on the audio, the McGurk Effect still occurs. Rosenblum suggests that this is evidence that once senses are integrated together, it is not possible to separate them.

Recent studies indicate that this integration occurs very early in the speech process, even before phonemes (the basic units of speech) are established.

Rosenblum suggests that physical movement of speech (that is, our mouths and lips moving) create acoustic and visual signals which have a similar form. He argues that as far as the speech brain is concerned, the auditory and visual information are never really separate. This could explain why we integrate speech so readily and in such a way that the audio and visual speech signals become indistinguishable from one another.

Rosenblum concludes that visual-speech research has a number of clinical implications, especially in the areas of autism, brain injury and schizophrenia and that "rehabilitation programs in each of these domains have incorporated visual-speech stimuli."

Similar Topics

1 - Hearing Charities of America's Hearing Aid Project Changing Lives with Donor's Help - Hearing Charities of America.
2 - Brain Imaging Predicts Language Learning in Deaf Children - Ann & Robert H. Lurie Children's Hospital of Chicago.
3 - Sign Language Comparative List of Astronomical Words - International Astronomical Union.
4 - Sign Language May Offer Answer to Meaning of Music - New York University.
5 - Human Brain Tunes Into Visual Rhythms In Sign Language - University of Chicago.
From our Deaf Communication section - Full List (63 Items)

Submit disability news, coming events, as well as assistive technology product news and reviews.


Loan Information for low income singles, families, seniors and disabled. Includes home, vehicle and personal loans.


Famous People with Disabilities - Well known people with disabilities and conditions who contributed to society.


List of awareness ribbon colors and their meaning. Also see our calendar of awareness dates.


Blood Pressure Chart - What should your blood pressure be, and information on blood group types/compatibility.



1 - Hearing Charities of America's Hearing Aid Project Changing Lives with Donor's Help
2 - A More Complete Mediterranean Diet May Protect Against Aggressive Prostate Cancer
3 - U.S. Mortality Rate Improvement Slowed by Rise in Obesity
4 - Delta Introduces Enhanced Requirements for Customers Traveling with Service or Support Animals



Citation


Disclaimer: This site does not employ and is not overseen by medical professionals. Content on Disabled World is not intended to be a substitute for professional medical advice, diagnosis, or treatment. Always seek the advice of a physician or other qualified health provider with any questions you may have regarding a medical condition. See our Terms of Service for more information.

Reporting Errors: Disabled World is an independent website, your assistance in reporting outdated or inaccurate information is appreciated. If you find an error please let us know.