Fighting virtual reality sickness

Columbia Engineering Professor Steven K. Feiner and Ajoy Fernandes MS'16 have developed a method of combating virtual reality (VR) sickness that can be applied to consumer head-worn VR displays, such as the Oculus Rift, HTC Vive, Sony PlayStation VR, and Google Cardboard. Their approach dynamically, yet subtly, changes the user's field of view (FOV) in response to visually perceived motion, as the user virtually traverses an environment while remaining physically stationary. The study showed that by strategically and automatically manipulating FOV, the degree of VR sickness experienced by participants can be significantly reduced. Furthermore, the researchers accomplished this without decreasing the participants' sense of presence in the virtual environment, and without the majority of the participants even being aware of the intervention.

The study was presented at IEEE 3DUI 2016 (IEEE Symposium on 3D User Interfaces) on March 20, where it won the Best Paper Award.

"2016 is the year of VR and it's estimated that over 200 million VR headsets will be sold by 2020," says Computer Science Professor Feiner, who directs the Computer Graphics and User Interfaces Lab. "But VR sickness, which has symptoms similar to motion sickness, poses a barrier for many users of this immersive technology. People who experience VR sickness will often stop using their headsets, as they feel nauseated and uncomfortable."

Feiner and Fernandes developed an approach that focused on the VR user's FOV, changing it in a way that many participants found imperceptible. "I was aware of work on change blindness—a concept in perceptual psychology that explains why people sometimes do not notice what would seem to be an obvious change in a scene," explains Fernandes, who himself has suffered from VR sickness. "So I wondered if this could be applied to VR: could we change a participant's FOV without them noticing?"

The team specifically targeted scenarios in which users move in the virtual environment in a way that intentionally differs from how they move in the real world. These include games in which they are physically standing or sitting on a couch in their living room, while walking, running, driving, or flying in the virtual world. In these scenarios, the visual motion cues that users see are at odds with the physical motion cues that they receive from their inner ears' vestibular system, the cues that provide us with our sense of motion, equilibrium, and spatial orientation. When the visual and vestibular cues conflict, users can feel quite uncomfortable, even nauseated.

The video will load shortly

In many cases, decreasing the field of view can decrease these symptoms. But decreasing FOV can also decrease the user's sense of presence in the virtual environment, making the experience less compelling. Feiner and Fernandes focused on subtly decreasing FOV in situations when a larger FOV would be likely to cause VR sickness (when the mismatch between physical and virtual motion increases) and restoring it when VR sickness is less likely to occur (when the mismatch decreases). They developed software that functions as a pair of "dynamic FOV restrictors" and can partially obscure each eye's view with a virtual soft-edged cutout. They then determined how much the user's field of view should be reduced, and the speed with which it should be reduced and then restored, and tested the system.

Their multi-day user study divided 30 voluntary participants into two groups. One group explored a VR environment without the dynamic FOV restrictors on one day and with the restrictors on a second day. This order was reversed for the second group. When study participants used the FOV restrictors, they felt more comfortable and stayed in the virtual environment longer than they did without the restrictors. Two kinds of restrictors were used, one of which was designed to be more subtle than the other. Most of the participants who used the more subtle restrictors did not notice them, and all those who did notice them said they would prefer to have them in future VR experiences.

Feiner and Fernandes plan next to look into how FOV restrictors could help acclimate users to VR experiences. For this study, they used soft-edged, circular cutouts and now want to explore using different cutout shapes and textures. They are also interested in going beyond their current study, in which they reduced FOV as a simple function of speed and angular velocity, to instead see the effect of reducing FOV based on parameters such as heart rate or optical flow.

"Virtual reality has the potential to profoundly change how we interact with people, machines, and information," Feiner says. "It is critical that the experience be both comfortable and compelling, and we think we've found a way."

Feiner and Fernandes' team, which has filed a provisional patent through Columbia Technology Ventures, included Luis Tolosa and Matthieu Gavaudan who helped coordinate and run the user study, and Carmine Elvezio and Mengu Sukan who gave feedback and suggestions.

Explore further: Oculus Rift brings a whole new dimension to communication

More information: S. Fernandes and S.K. Feiner, "Combating VR sickness through subtle dynamic field-of-view modification," 2016 IEEE Symposium on 3D User Interfaces (3DUI), Greenville, SC, USA, 2016, pp. 201-210. DOI: 10.1109/3DUI.2016.7460053