Skip to main content
Accessibility|Contact|Privacy|Terms|Cookies

Google Glass App - Real Time Closed Captioning

  • Published: 2014-10-11 (Revised/Updated 2016-09-23) : Author: Georgia Institute of Technology : Contact: cog.gatech.edu
  • Synopsis: Researchers have created speech-to-text software for Google Glass that helps hard-of-hearing with everyday conversations.

Quote: "If people with hard-of-hearing understand the speech, the conversation can continue immediately without waiting for the caption."

Main Document

A person with hard-of-hearing wears Google Glass while a second person speaks directly into a smartphone. The speech is converted to text, sent to Glass and displayed on its heads-up display.

Captioning on Glass uses a companion Android phone app for everyday conversations. A person with hard-of-hearing wears Glass while a second person speaks directly into a smartphone. The speech is converted to text, sent to Glass and displayed on the heads-up display.

A group in Georgia Tech's College of Computing created the Glassware when one of its own said he was having trouble hearing and thought Glass could help him.

"This system allows wearers like me to focus on the speaker's lips and facial gestures," said School of Interactive Computing Professor Jim Foley. "If people with hard-of-hearing understand the speech, the conversation can continue immediately without waiting for the caption. However, if I miss a word, I can glance at the transcription, get the word or two I need and get back into the conversation."

Foley's colleague, Professor Thad Starner, leads the Contextual Computing Group working on the project. He says using a smartphone with Glass has several benefits as compared to using Glass by itself.

"Glass has its own microphone, but it's designed for the wearer," said Starner, who is also a technical lead for Glass. "The mobile phone puts a microphone directly next to the speaker's mouth, reducing background noise and helping to eliminate errors."

Starner says the phone-to-Glass system is helpful because speakers are more likely to construct their sentences more clearly, avoiding "uhs" and "ums." However, if captioning errors are sent to Glass, the smartphone software also allows the speaker to edit the mistakes, sending the changes to the person wearing the device.

"The smartphone uses the Android transcription API to convert the audio to text," said Jay Zuerndorfer, the Georgia Tech Computer Science graduate student who developed the software. "The text is then streamed to Glass in real time."

A hard-of-hearing person wears Glass while a second person speaks directly into a smartphone
About This Image: A hard-of-hearing person wears Glass while a second person speaks directly into a smartphone
Captioning on Glass is currently available to install from MyGlass. More information and support can be found at the project website.

Foley and the students are working with the Association of Late Deafened Adults in Atlanta to improve the program.

The same group is also working on a second project, Translation on Glass, that uses the same smartphone-Glass Bluetooth connection process to capture sentences spoken into the smartphone, translate them to another language and send them to Glass.

The only difference is that the person wearing Glass, after reading the translation, can reply.

The response is translated back to the original language on the smartphone.

Two-way translations are currently available for English, Spanish, French, Russian, Korean and Japanese.

"For both uses, the person wearing Glass has to hand their smartphone to someone else to begin a conversation," said Starner. "It's not ideal for strangers, but we designed the program to be used among friends, trusted acquaintances or while making purchases."

The group is working to get Translation on Glass ready for the public.


Have Your Say! - Add your comment or discuss this article on our FaceBook Page.


Interesting Similar Topics
1 : Screen Reader Plus Keyboard Helps Blind, Low-Vision Users Browse Modern Webpages : University of Washington.
2 : HopkinsPD Smartphone App Helps Doctors Track Severity of Parkinson's Disease Symptoms : Johns Hopkins University.
3 : Cleen App Aims to Improve UK Public Washroom Standards and Accessibility : Sarah Jenkin-Jones.
4 : Libby App Enables Free Ebooks and Audiobooks From the Library : Rakuten OverDrive.
5 : Mobile Phone App to Control Diabetes May be Possible Someday : Rutgers University.
From our Disability and Health Apps section - Full List (116 Items)


Submit disability news, coming events, as well as assistive technology product news and reviews.


Loan Information for low income singles, families, seniors and disabled. Includes home, vehicle and personal loans.


Famous People with Disabilities - Well known people with disabilities and conditions who contributed to society.


List of awareness ribbon colors and their meaning. Also see our calendar of awareness dates.


Blood Pressure Chart - What should your blood pressure be, and information on blood group types/compatibility.





1 : Eating at Night, Sleeping By Day Alters Key Blood Proteins
2 : Interior Car Temperature Can Become Life-threatening for Children in an Hour
3 : 20 New Episodes of Letters to Lynette with Dr. Lynette Louise to Air on The Autism Channel in 2018
4 : Turnstone Center Designated as Official Paralympic Training Site by US Olympic Committee
5 : Help Your Child in School by Adding Language to The Math
6 : 50% of Retirees Saw Little or No COLA Increase in Net 2018 Social Security Benefits
7 : Turnstone Endeavor Games Concludes with National Records Broken
8 : Spinning in Circles and Learning From Myself by Tsara Shelton


Disclaimer: This site does not employ and is not overseen by medical professionals. Content on Disabled World is not intended to be a substitute for professional medical advice, diagnosis, or treatment. Always seek the advice of a physician or other qualified health provider with any questions you may have regarding a medical condition. See our Terms of Service for more information.

Reporting Errors: Disabled World is an independent website, your assistance in reporting outdated or inaccurate information is appreciated. If you find an error please let us know.

© 2004 - 2018 Disabled World™