When someone pulls on a virtual reality headset, they’re ready to dive into a simulated world. They might be hanging out in VRChat or slashing beats in Beat Saber. Regardless, interacting with that world usually involves hand controllers. But new virtual reality — or VR — technology out of Australia is hands-free. Facial expressions allow users to interact with the virtual environment. This technology would allow people who can’t use handheld controllers to play virtual games.
This setup could make virtual worlds more accessible to people who can’t use their hands, says Arindam Dey. He studies human-computer interaction at the University of Queensland in Brisbane. Other hands-free VR tech has let people move through virtual worlds by using treadmills and eye-trackers. But not all people can walk on a treadmill. And most people find it a challenge to stare at one spot long enough for the VR system to register the action. Simply making faces may be an easier way for those who are disabled to navigate VR.
In the researchers’ new system, VR users wear a cap studded with sensors. Those sensors record brain activity. The sensors can also pick up facial movements that signal certain expressions. Facial data can then be used to control the user’s movement through a virtual world.
Facial expressions usually signal emotions. So Dey’s team designed three virtual environments for users to explore. An environment called “happy” required participants to catch butterflies with a virtual net. “Neutral” had them picking up items in a workshop. And in the “scary” one, they had to shoot zombies. These environments allowed the researchers to see whether situations designed to provoke certain emotions affected someone’s ability to control VR through expressions.
Eighteen young adults tested out the technology. Half of them learned to use three facial expressions to move through the virtual worlds. A smile walked them forward. A frown brought them to a stop. And to perform a task, they clenched their teeth. In the happy world, that task was swooping a net. In the neutral environment, it was picking up an item. In the scary world, it was shooting a zombie.
The other half of participants interacted with the virtual worlds using hand controllers. This was the control group. It allowed the researchers to compare use of facial expressions with the more common form of VR interaction.
Using facial expressions made participants feel more present inside the virtual worlds. But expressions were more challenging to use than hand controllers. Recordings from the sensor-laden cap showed that the brains of people using facial expressions were working harder than those who used hand controllers. But that could just be because these people were learning a new way to interact in VR. Perhaps the facial expression method would get easier with time. Importantly, virtual settings meant to trigger different emotions did not affect someone’s ability to control their VR using facial expressions.
