Challenges with Virtual Reality

Consumer-grade virtual reality is steadily getting lighter, cheaper, and more polished, but it still has a number of technical hurdles to overcome to truly reach its mass consumer potential. Fortunately, renewed interest in virtual reality the past few years has led to an influx of investments in the field, which should accelerate uncovering solutions. The following information explores some of the major issues facing virtual reality today and how some companies are working to resolve them.

Virtual reality challenge 1: Simulator sickness

Early HMD devices generated widespread user complaints of motion sickness. This issue alone was enough to derail early virtual reality mass consumer devices such as Sega virtual reality and Nintendo’s Virtual Boy. It’s an issue that modern headset manufacturers still grapple with.

Motion sickness can occur when there are inconsistent signals between your inner ear’s vestibular sense of motion and what your eyes are seeing. Because your brain senses these signals are inconsistent, it then assumes that your body is sick, potentially from a toxin or some other affliction. At that point, your brain may decide to induce headaches, dizziness, disorientation, and nausea. Virtual reality headset use can induce a type of motion sickness that doesn’t necessarily involve any real motion; researchers have dubbed it simulator sickness.

There are a number of ways to combat simulator sickness, including some unconventional approaches. One study by Purdue University’s Department of Computer Graphics Technology suggested adding a “virtual nose” to every virtual reality application to add a stabilizing effect for the user. Virtualis LLC is commercializing this virtual nose, naming it nasum virtualis. Embedding the nose in the user’s field of view acts as a fixed point of reference to ease virtual reality sickness. Your physical nose appears in your line of sight in real life, but you often don’t perceive it there. Similarly, most users in Virtualis’s studies didn’t even notice the virtual nose in virtual reality, but they reported a 13.5 percent drop in severity of sickness and an increase of time spent in the simulator.

This image displays how a virtual nose may appear to users in virtual reality.

The most effective way that virtual reality developers can combat simulator sickness, however, is to minimize latency between a user’s physical motion and the headset’s response. In the real world, there is no latency between the movement of our heads and the visual response of the world around us. Reproducing that lack of latency in headset is of paramount importance.

With the proliferation of virtual reality on low-powered mobile devices, it’s more important than ever to keep the frames per second (FPS) each headset can display as high as possible. Doing so enables the visuals in the headset to stay aligned with the user’s movement.

Here are some other suggestions for avoiding simulator sickness when creating or using virtual reality apps:

• Ensure the headset is properly adjusted. If the virtual world seems blurry when trying on your headset, most likely the headset needs adjustment. Most headsets enable users to tweak the device’s fit and distance from their eyes to eliminate any blurriness. Be sure your headset is properly adjusted before going into any virtual reality experience.
• Sit down. For some people, the sense of stability that being seated provides helps them overcome their motion sickness.
• Keep text legible. Avoid reading or using small-font text in virtual reality, and keep text use to a minimum (just a few words at a time).
• No unexpected movement. When developing, never move the camera programmatically without reason. The user should feel motion occurs due to his own motion or triggered via interaction.
• Avoid acceleration. It is possible to move virtual cameras without triggering simulator sickness, but that motion needs to be smooth. Avoid accelerating or decelerating a user when motion in the virtual space needs to occur.
• Always track the user’s movements. Do not reverse the camera against a user’s motion or stop tracking the user’s head position. A user’s view must update with her head movement.
• Avoid fixed-view items. Fixed-view items (items that do not change when the view changes, such as a pop-up to inform the user of something in the middle of the screen or a heads-up display [HUD>) are fairly common in 2D gaming today. This mechanic does not work well in virtual reality, however, because it isn’t something users are accustomed to in the real world.

As devices become more powerful, simulator sickness due to low FPS should, in theory, become less common. The more powerful the device, the better the ability to keep the visuals and movement of the virtual world tracked to the user’s physical movement, reducing the main cause of simulator sickness.

However, although we have far surpassed the processing power of computers of old, we still run into software that often runs slower than games of 20 years ago. In general, the more powerful our hardware gets, the more we tend to ask of it. Better visuals! More items on screen! Larger fields of view! More effects! Knowing the potential causes of simulator sickness should help you navigate these issues should they appear.

Virtual reality challenge 2: The screen-door effect

Put on any older virtual reality headsets, or some of the current smartphone-powered virtual reality headsets, and look closely at the image produced in the headset. Depending on the resolution of the device you’re using, you may notice “lines” in between the displayed pixels. As a child, you may have noticed the same thing if you sat too closely to your older TV set. This issue is called the screen-door effect, for its resemblance to looking at the world through a screen door. Although this problem has long been solved for televisions of today with extremely high resolutions, it has been reintroduced in some virtual reality headsets.

The image below displays an example of an exaggerated screen-door effect example as seen in virtual reality. (An actual screen-door effect would occur on a pixel-by-pixel basis, not shown here.) With the display so close to a user’s face, the grid of space between pixels can begin to become apparent.

This effect is most noticeable in displays with lower resolutions, such as older headsets or some smartphones, many of which were never intended to be used primarily as virtual reality machines, held inches from your nose and magnified via optic lenses.

Various proposals have been put forth to solve this issue. For example, LG has suggested placing a “light diffuser” between the screen and lenses, though most people agree the real fix will be higher-resolution displays. High-definition displays should alleviate the screen-door effect, as they have for TV, but they’ll require more processing power to run. As with simulator sickness, the hope is that the better the hardware, the less likely this effect will be to occur. With a little luck, the screen-door effect should become a relic of the past within the next generation or two of virtual reality headsets.

Virtual reality Challenge 3: Movement in VIRTUAL REALITY

Moving through the digital environment of virtual reality is still an issue. Higher-end headsets, such as Vive and Rift, allow users to be tracked throughout a room, but not much farther than that. Anything more requires some sort of movement mechanic built into the application itself, or specialized hardware beyond what most consumers likely have available.

Movement over large distances in virtual reality will likely be an ongoing logistical problem for application developers. Even utilizing some of the solutions listed previously, movement in virtual reality that does not correspond to physical motion can trigger simulator sickness in some users. And even if you could guarantee an omni-directional treadmill for every user to track his or her movement, there are often large distances users will not want to cover walking. Plus, users with limited mobility may be unable to traverse distances on foot. Movement is an issue that hardware and software developers will need to work together to solve. And there are solutions available.

Virtual reality Challenge 4: Health effects

Health risks are perhaps the largest unknown on this list. The Health and Safety guidelines for Oculus Rift caution against use if the user is pregnant, elderly, tired, or suffering from heart conditions. They also warn that users might experience severe dizziness, seizures, or blackouts. Scary-sounding stuff! There are also big unknowns as to the long-term health effects of virtual reality. Researchers have yet to thoroughly study the impact of long-term use of virtual reality headsets on eyesight and the brain.

Initial studies have generally shown that most adverse health effects are short-term, with little lasting effect on the user. However, as users begin to stay in virtual reality space for longer stretches at a time, further studies would be required to discover any long-term effects of virtual reality usage.

In the meantime, virtual reality companies seem to be erring on the side of caution regarding the potential long-term effects. As Sarah Sharples, president of the Chartered Institute of Ergonomics and Human Factors, said in an interview with The Guardian, “Absolutely there are potentially negative effects of using virtual reality. The most important thing that we should do is just to be cautious and sensible. But we should not let that stop us from taking advantage of the massive potential this technology offers as well.”

Virtual reality challenge 5: Cannibalization of the market

One final worry has to do with the virtual reality market as a whole. The mobile market (and specifically the cheapest implementation, Google Cardboard) holds a massive adoption advantage over the higher-end headsets (refer to Table 2-3). And perhaps for good reason. It’s easier for a consumer to stomach a $20 purchase of a low-end mobile virtual reality headset than to save up a few hundred dollars for a higher-end model.

Predictably, low-end headsets tend to offer lower-end experiences. A user may dismiss a low-end virtual reality system as little more than a toy, believing it represents the current level of virtual reality immersion, when nothing could be farther from the truth. However, perception can often become reality. Could the proliferation of low-end virtual reality implementations serve to harm virtual reality’s adoption in the long run, cannibalizing its own market?

The sales figures of lower-end devices have likely made some companies sit up and take notice. Many manufacturers appear to be focusing on a multi-tiered strategy for their next generation of headsets, offering experiences ranging from low end to high end for consumers. For example, Oculus co-founder Nate Mitchell claimed in an interview with Ars Technica that Oculus would focus on a three-headset strategy for its next generation of consumer headsets, with the standalone Oculus Go, set to be released in 2018, as its lower-end standalone device, followed by the Oculus Santa Cruz, a mid-tier headset experience. Similarly, HTC has released the HTC Vive Pro as a higher-end device, with the standalone HTC Vive Focus (released in China), focused more on the mid-tier market.

In the long run, there is likely a broad enough market base to support all variations of virtual reality. With the advent of the next generation of headsets, it will be interesting to see which devices make the biggest inroads with consumers. The near future will likely bring a rise in midrange mobile device headsets, while the higher-end, external PC-based headsets cater to those going all-in on high-end devices. The latter is a smaller demographic, but one that is willing to spend more to get the best-in-class experience that virtual reality can offer.