New VR Technology Uses Mouth Gestures to Interact in Virtual Reality
Topic: Electronics/Software
Author: Binghamton University - Contact: binghamton.edu
Published: 2017/10/05
Contents: Summary - Introduction - Main Item - Related Topics
Synopsis: New VR framework interprets mouth gestures as a medium for interaction within virtual reality in real-time.
Introduction
Researchers at Binghamton University, State University of New York have developed a new technology that allows users to interact in a virtual reality environment using only mouth gestures.
Main Item
The proliferation of affordable virtual reality head-mounted displays provides users with realistic immersive visual experiences. However, head-mounted displays occlude the upper half of a user's face and prevent facial action recognition from the entire face. To combat this issue, Binghamton University Professor of Computer Science Lijun Yin and his team created a new framework that interprets mouth gestures as a medium for interaction within virtual reality in real-time.
Yin's team tested the application on a group of graduate students.
Once a user put on a head-mounted display, they were presented with a simplistic game; the objective of the game was to guide the player's avatar around a forest and eat as many cakes as possible. Players had to select their movement direction using head rotation, move using mouth gestures and could only eat the cake by smiling. The system was able to describe and classify the user's mouth movements, and it achieved high correct recognition rates. The system has also been demonstrated and validated through a real-time virtual reality application.
"We hope to make this applicable to more than one person, maybe two. Think Skype interviews and communication," said Yin. "Imagine if it felt like you were in the same geometric space, face to face, and the computer program can efficiently depict your facial expressions and replicate them so it looks real."
Though the tech is still in the prototype phase, Yin believes his technology is applicable to a plethora of fields.
"The virtual world isn't only for entertainment. For instance, health care uses VR to help disabled patients," said Yin. "Medical professionals or even military personal can go through training exercises that may not be possible to experience in real life. This technology allows the experience to be more realistic."
Students Umur Aybars Ciftci and Xing Zhang contributed to this research.
The paper, "Partially occluded facial action recognition and interaction in virtual reality applications," was presented at the 2017 IEEE International Conference on Multimedia and Expo.
Explore Related Topics
1 - Advanced Touch-Feedback System Creates Consistent Tactile Experience on Screens - Consistent virtual haptic technology for virtual reality (VR) and augmented reality (AR) users.
2 - HeardAI Making Voice Activated Artificial Intelligence Accessible - HeardAI has advanced to Phase 2 of National Science Foundation's Convergence Accelerator program to make voice-activated AI accessible and fair to people who stutter.
3 - A11yBoard Google Slides Extension Makes Presentation Software More Accessible for the Blind - A11yBoard for Google Slides is a browser extension and phone app that allows blind users to navigate through complex slide layouts and text.
4 - Neuromorphic Computer Chip Mimics Human Vision and Memory Abilities - The neuromorphic invention is a single chip enabled by a sensing element, doped indium oxide, that's thousands of times thinner than a human hair and requires no external parts to operate.
5 - Neural Prosthesis: Using Brain Activity to Decode Speech - Speech decoding from a small set of spatially segregated minimally invasive intracranial EEG electrodes with a compact and interpretable neural network.
Page Information, Citing and Disclaimer
Disabled World is a comprehensive online resource that provides information and news related to disabilities, assistive technologies, and accessibility issues. Founded in 2004 our website covers a wide range of topics, including disability rights, healthcare, education, employment, and independent living, with the goal of supporting the disability community and their families.
Cite This Page (APA): Binghamton University. (2017, October 5). New VR Technology Uses Mouth Gestures to Interact in Virtual Reality. Disabled World. Retrieved October 6, 2024 from www.disabled-world.com/assistivedevices/computer/mouth-gestures.php
Permalink: <a href="https://www.disabled-world.com/assistivedevices/computer/mouth-gestures.php">New VR Technology Uses Mouth Gestures to Interact in Virtual Reality</a>: New VR framework interprets mouth gestures as a medium for interaction within virtual reality in real-time.
Disabled World provides general information only. Materials presented are never meant to substitute for qualified medical care. Any 3rd party offering or advertising does not constitute an endorsement.