Comments made by Oculus CTO John Carmack at Oculus Connect 5 and Twitter suggest that the upcoming Oculus Rift S cameras could support finger tracking.
To be clear, this doesn’t mean the software to do so would be ready by the time Rift S launches. But it does mean the headset could one day recieve the feature as a software update in future- the cameras are seemingly suitable.
Rift S Onboard Cameras
Rift S was first revealed in a TechCrunch report in October. The report revealed the headset would be an iterative update, increasing resolution and changing to the same inside-out tracking system as the upcoming Oculus Quest VR console.
Earlier this month we confirmed the TechCrunch report by discovering references in the Oculus PC software code to a ‘Rift S’ with onboard cameras.
Carmack’s Comments
During 2018’s Oculus Connect 5 conference, Oculus CTO John Carmack briefly spoke about a project to bring finger tracking to Oculus Quest:
This seemed to suggest that the main limitation for finger tracking on the headset was the power & compute limitations of the mobile platform. He stated that Quest can be used as a platform to research finger tracking for future headsets with “the power and ability to do real time”.
The TechCrunch report and our findings suggest that Rift S will feature the same Insight cameras as Quest. So given the enormous relative power of a PC, shouldn’t Rift S be capable of finger tracking?
To verify we weren’t misunderstanding his comments, we reached out to Carmack on Twitter. Here’s how he responded:
Carmack doubled down on stressing the power limitations of mobile as the main barrier. When asked about the featue in regards to PC, he confirmed it was possible. Requiring a reserved CPU core or two rules out this tech being usable in CPU-intensive apps. But it would be entirely possible for developers of apps suiting finger tracking to optimize for this limitation.
These comments aren’t indicative of the feature actually being available at launch- or ever. But what they do suggest is that the camera hardware is suitable and PCs are capable.
A Long Researched, Challenging Feature
The first indication of Facebook’s interest in finger tracking for VR was revealed in late 2014. The company acquired startup Nimble VR, composed of four veterans of hand tracking technology. But upon this acquisition, Oculus was clear that this tecnology “may not even be used in the CV2 or CV3”.
That’s likely because Nimble’s tech wasn’t just software, but a dedicated depth camera. Depth cameras deliver excellent finger tracking such as that found in the new HoloLens 2 AR headset. But the high end solutions are costly and the low end ones are bulky and relatively heavy, adding weight at the worst possible area of a headset (directly in front).
Delivering high quality finger tracking on regular cameras is an entirely different level of challenge. However, if it can be done it allows finger tracking to be added at no extra hardware cost to headsets which already use cameras for other tracking.
In 2016 at Oculus Connect 3, Chief Scientist Michael Abrash made a range of predictions about VR in the year 2021. He noted that finger tracking could be done perfectly with markers on gloves (and showed this off), but not yet directly. However, he predicted that by 2021, it would be possible:
Source: Oculus Rift S Cameras Could Support Finger Tracking In Future
