GUEST:
Interfacing with our digital world through VR and AR poses many challenges to interface designers. One of the biggest problems is one that not a lot of people are discussing: straightforward text entry.
I’ve been working in VR for years. In late 2013 I was reintroduced to the concept of VR by a startup from Irvine, California called Oculus. As part of my work at Sidekick Games, we were called by new platform makers to see their new inventions and suggest games that would feature the uniqueness of their new products. By then, I had the pleasure of working with PrimeSense, Intel, Leap Motion and other innovative companies who wanted to change how we interface with the digital world — which has also helped me see the challenges in VR interfaces.
Since 2013, when Oculus managed to sweep the world, over $5.5 billion has been invested in VR and AR. Much of this investment has gone to make better wearable screens and better GPUs and CPUs, critical to present the visuals to our eyes at a rate that would feel comfortable and real.
The importance of precision
Precise controls — the input part of the input-CPU-output chain — have been neglected. Maybe the lack of popularity of body-tracking products like Kinect or the early Leap Motion controls, as well as priority placed on creating better presentation units, have made input devices for VR a lower priority. Hardware companies have tried to cover interactivity by offering a simple one-button solution or, over 2016-2017, pointers with 3 degrees of freedom (3DoF) and multi-button 6DoF controllers such as the Oculus Touch. These controllers were made for gaming purposes or rough creations, using apps like Tilt Brush. Some solutions did try to recognize users’ hands, providing almost skeletal data or focusing on a set number of gestures, but very few have delivered the speed and resolution users need.
The lack of precise, fast input coupled with users’ attachment to their QWERTY keyboards creates a gap in input technology and has led to a lack of commercial text and numeric entry solutions. Unable to see the keys or rethink text entry, users are missing a critically needed part of interactivity : easy text entry. VR is similar to phones a decade or more ago, before full keyboards were introduced on touch screens, in that we’re looking for a way to easily enter text. With no big, clear buttons, we seem lost.
Not being able to easily enter text leads to a major lack of utility for VR apps. Text entry to write, search, fill forms, enter data, and perform calculations is critical, and without it, a large number of non-entertainment apps in VR are left behind. Without these utility apps, VR will always lack wide market adoption. Not having an easy text entry method in VR is forcing VR into its current entertainment and simulation platform status, blocking it from being a real utility that people can use as an alternative to their current computers.
As an example, a popular idea when people are introduced to VR is having a number of virtual “screens” around a user, which provide users with wealth of information and control options without expensive real-world screens. However, the lack of detailed input — specifically, an easy way to enter text and numbers in VR — has been a big obstacle to making that concept real. Instead, we’ve had to settle for floating keyboards with only a pointer, at best, to choose letters, one by one, similar to smart TVs’ keyboard.
The challenge of typing
The core challenge is this: being able to recognize a user’s hands down to the fingers level, and track movement fast enough to pick up typing gestures. The two main methods for solving this are first, using front-facing cameras, like Leap Motion, Pebbles, and uSens; and second, employing physical wearable trackers, such as the Oculus Glove or connected rings. Recently Logitech announced its own solution, a physical keyboard that is recognizable in virtual spaces and which should provide a relief for QWERTY keyboard-hungry users.
Another concept is getting rid of buttons all together. Tap Systems offers a wearable keyboard solution that tracks your fingers’ taps and maps them into characters and numbers. Tap offers a totally new way to think about text entry. While you are still using your fingers to tap, you can tap anywhere, without the limits of buttons with specific characters on them. Just like touch typing, you memorize the number of fingers to tap, on any surface, to get the character you need. The result is fluid typing without a need to look at or have a keyboard.
Voice controls, becoming popular with Siri, Alexa, and Google (aka OK Goo Goo), are maturing but still slow, not discreet, and tough on accents.
The QWERTY conundrum
Anyone 15 years old and above is addicted to QWERTY keyboards. It is not impossible to have people learn new ways to type, however, as we have seen with feature phones and in other niche markets. QWERTY is known to everyone, but it’s also a system that was made to accommodate the limitations physical typewriters. It is not optimized or the fastest solution. This is why I’m excited about new, better ways to enter text. I’m also excited about how younger people, ages 5 or 6, can start fresh with another, or an additional, way to enter text. VR, as a new platform, can play an important role in that switch from the old keyboard concept to the new.
Whatever solution will prevail, whether it is the old QWERTY keyboard reimagined or a whole new concept of typing, text entry is critical part of interaction with our expanding digital world and will remain a barrier to wider market adoption of VR until it is solved.
Guy Ben-Dov is an interactive entertainment entrepreneur who has worked with innovative teams in Fortune 500 companies as well as startups all around the world.
