Skip to main content
Accessibility|Contact|Privacy|Terms|Cookies

New VR Technology Uses Mouth Gestures to Interact in Virtual Reality

  • Published: 2017-10-05 : Author: Binghamton University : Contact: binghamton.edu
  • Synopsis: New VR framework interprets mouth gestures as a medium for interaction within virtual reality in real-time.

Quote: "The virtual world isn't only for entertainment. For instance, health care uses VR to help disabled patients..."

Main Document

Researchers at Binghamton University, State University of New York have developed a new technology that allows users to interact in a virtual reality environment using only mouth gestures.

The proliferation of affordable virtual reality head-mounted displays provides users with realistic immersive visual experiences. However, head-mounted displays occlude the upper half of a user's face and prevent facial action recognition from the entire face. To combat this issue, Binghamton University Professor of Computer Science Lijun Yin and his team created a new framework that interprets mouth gestures as a medium for interaction within virtual reality in real-time.

Yin's team tested the application on a group of graduate students.

Binghamton University Professor of Computer Science Lijun Yin and his team created a new framework that interprets mouth gestures as a medium for interaction within virtual reality in real-time. Photo Credit: Binghamton University, State University of New York
About This Image: Binghamton University Professor of Computer Science Lijun Yin and his team created a new framework that interprets mouth gestures as a medium for interaction within virtual reality in real-time. Photo Credit: Binghamton University, State University of New York
Once a user put on a head-mounted display, they were presented with a simplistic game; the objective of the game was to guide the player's avatar around a forest and eat as many cakes as possible. Players had to select their movement direction using head rotation, move using mouth gestures and could only eat the cake by smiling. The system was able to describe and classify the user's mouth movements, and it achieved high correct recognition rates. The system has also been demonstrated and validated through a real-time virtual reality application.

"We hope to make this applicable to more than one person, maybe two. Think Skype interviews and communication," said Yin. "Imagine if it felt like you were in the same geometric space, face to face, and the computer program can efficiently depict your facial expressions and replicate them so it looks real."

Though the tech is still in the prototype phase, Yin believes his technology is applicable to a plethora of fields.

"The virtual world isn't only for entertainment. For instance, health care uses VR to help disabled patients," said Yin. "Medical professionals or even military personal can go through training exercises that may not be possible to experience in real life. This technology allows the experience to be more realistic."

Students Umur Aybars Ciftci and Xing Zhang contributed to this research.

The paper, "Partially occluded facial action recognition and interaction in virtual reality applications," was presented at the 2017 IEEE International Conference on Multimedia and Expo.

Discussion

• Have your say! Add your comment or discuss this article on our FaceBook Page.

Similar Topics

1 : Epihunter Classroom - Making Silent Epileptic Seizures Visible : Mark Winters.
2 : Access Stylus Makes Touchscreens Accessible for Everyone : Arnoldware Applications.
3 : GlassOuse Assistive Device V1.2 and G-Switch Series are Ready for Pre-order : GlassOuse Assistive Devices.
4 : Wearable Stretchable Electronics Aid Stroke Recovery Treatment : Northwestern University.
5 : Augmented Reality ProjectDR Lets Doctors See Under the Skin : University of Alberta.
From our Electronic Devices and Software section - Full List (182 Items)


Submit disability news, coming events, as well as assistive technology product news and reviews.


Loan Information for low income singles, families, seniors and disabled. Includes home, vehicle and personal loans.


Famous People with Disabilities - Well known people with disabilities and conditions who contributed to society.


List of awareness ribbon colors and their meaning. Also see our calendar of awareness dates.


Blood Pressure Chart - What should your blood pressure be, and information on blood group types/compatibility.





1 : Help Your Child in School by Adding Language to The Math
2 : 50% of Retirees Saw Little or No COLA Increase in Net 2018 Social Security Benefits
3 : Turnstone Endeavor Games Concludes with National Records Broken
4 : Spinning in Circles and Learning From Myself by Tsara Shelton
5 : St. Louis HELP Medical Equipment Donation Drive Generates Record-Breaking Results
6 : People Who Snore Suffer from Palate Nerve and Muscle Damage
7 : How Our Ancestors with Autistic Traits Led a Revolution in Ice Age Art
8 : Housing and Disabled People: Britains Hidden Crisis


Disclaimer: This site does not employ and is not overseen by medical professionals. Content on Disabled World is not intended to be a substitute for professional medical advice, diagnosis, or treatment. Always seek the advice of a physician or other qualified health provider with any questions you may have regarding a medical condition. See our Terms of Service for more information.

Reporting Errors: Disabled World is an independent website, your assistance in reporting outdated or inaccurate information is appreciated. If you find an error please let us know.

© 2004 - 2018 Disabled World™