Watch: How ‘Star Wars: Rogue One’ Used Steam VR Tracking to Shoot VFX Shots

It’s emerged that last year’s Star Wars: Rogue One, used existing virtual reality technology – specifically the HTC Vive and Steam VR – to allow the film’s director to compose visual effects shots for the movie.

Whatever you may have thought of the most recent arrival to the Star Wars movie universe, Rogue One had some undeniably impressive visual effects. Those effects have since been nominated for 2 2017 Academy Awards and, thanks to a recent episode of BBC Click, it’s also been revealed that virtual reality technology played a significant role in the direction of them.

If anyone has any interest in visual effects, they’ll be aware of a system called Animatics. It’s a way to digitally storyboard movie sequences, often those which are technically complex or that involve VFX shots in some way, and also to convey (often crudely) shot composition and movement in the scene in question. Animatics has been used for many years to this end, but ultimately the scene being shot will still ultimately shot traditionally. Later came example of directors who embraced realtime Animatics fused with motion capture in order to direct VFX shots using a form of augmented reality. Peter Jackson used such a technique to direct VFX heavy scenes from his Lord of the Rings trilogy for example.

Now, it seems that Star Wars: Rogue One Director Gareth Edwards used off-the-shelf virtual reality technology, specifically Steam VR tracking as used with the HTC Vive, to position and compose some of the epic VFX shots from the movie.

In this video (above) the BBC Click team visited ILM (Industrial Light and Magic) London, who revealed their “VCam Renderer” system and how it was used in the making of the film. The system, which comprises a tablet computer and a Steam VR (Vive) controller strapped to the back, leverages Lighthouse positional tracking to translate its position in real world space to that of a virtual camera, shooting the scene.

Ellis, speaking about why the system was employed for the film, says it was down to Director Edwards’ style. “So he’s a very ‘hands on’ film maker, he likes to walk around his sets and sort of physically pick up the cameras himself and walk around and find interesting angles that might not have occurred to him when he was planning out the shoots in pre-production.” So, the VCam system was born, allowing Edwards to see realtime Animatic of VFX scenes and then for him to position the virtual camera within those scenes, in order that the VFX team could execute the final shot according to his vision.

It’s a testament to the robust accuracy of Valve’s Lighthouse tracking system that Hollywood studios feel confident in adopting this off-the-shelf hardware in making their multi-million dollar blockbusters. If anyone knows of other instances where the technique has been used (or is being used), we’d love to hear about it in the comments below.