Now Available – Light-field Captured ‘Hallelujah’ is a Stunning Mix of Volumetric Film and Audio

Hallelujah is a new experience by VR film studio Within that’s captured using Lytro’s latest Immerge light-field camera which captures volumetric footage that makes for a much more immersive experience than traditional 360 video. Hallelujah is a performance of Leonard Cohen’s 1984 song of the same name, and mixes the latest in VR film capture technology with superb spatial audio to form a stunning experience.

Update (9/22/17): The light-field captured piece, Hallelujah, is finally available to the public through the Within app on just about every mobile and desktop VR platform, for free. Head to the Within website to be directed to the app for your platform of choice.

Original Article (4/23/17): Lytro’s Immerge camera is unlike any 360 camera you’ve seen before. Instead of shooting individual ‘flat’ frames, the Immerge camera has a huge array of cameras which gather many views of the same scene, data which is crunched by special software to recreate the actual shape of the environment around the camera. The big benefit of which is that the playback puts the viewer in a virtual capture of the space, allowing for a limited amount of movement within the scene, whereas traditional 360 video only captures a static viewpoint which is essentially stuck to your head. Not to mention the Immerge camera also provides true stereo and outputs a much higher playback quality. The result is a much richer and more immersive VR film experience than what you’ve seen with traditional 360 video shoots.

After a recent visit to Lytro check out the latest version of the Immerge camera, I concluded, “All-in-all, seeing Lytro’s latest work with Immerge has further convinced me that today’s de facto 360-degree film capture is a stopgap. When it comes to cinematic VR film production, volumetric capture is the future, and Lytro is on the bleeding edge.”

In that article I talked a lot about the tech, but I couldn’t say much about the experience that I saw which left me so impressed. Hallelujah was that experience, and now I can talk about it.

Created by VR film studio Within, Hallelujah pairs Immerge’s volumetric capture capabilities with finely mixed spatial audio that forms the foundation for a stunning performance of Leonard Cohen’s 1984 song Hallelujah (perhaps most famously performed by Jeff Buckley). Lytro has provided a great behind-the-scenes look at the production here:

In Hallelujah, singer Bobby Halvorson starts as the solo lead of the song directly in front of the viewer against a pitch black background. As the only object in the scene, it’s easy to feel the volume and shape of the singer thanks to the volumetric capture. As you lean left and right, you see the sides of Halvorson’s face in a way that would be utterly impossible with traditional 360 capture techniques. With that sense of depth and parallax comes a feeling that the singer is really there right in front of you.

The entire song is sung a capella with no instrumentation, and copies of Halvorson are duplicated to the left, right, and behind the viewer, singing accompanying tracks.

There you are, in a void of blackness, with Halvorsons at arms length on all sides, singing a stirring version of the uplifting song. You can look all around you to see each version of Halvorson singing a different track as he creates all the notes of the song. As you turn your head, the careful audio mixing becomes apparent; each track accurately sounds like it’s coming from each of the respective singers. This excellently mixed spatial audio significantly enhanced the sense of Presence for me; Halvorson doesn’t just look and feel like he’s there in front of you, he also sounds like it.

As the song progresses, you may noticed a distinct and fitting reverb coming from Halvorson’s voice. It isn’t a digitally applied effect though; as the song progresses, the black void around him eventually fades away and you find yourself in the midst of a beautifully adorned church. Behind the singer is a full church choir that joins into the song all at once, adding a swelling of new voices to Halvorson’s self-accompanied solo.

Turning around to explore this newly unveiled environment, you can see the ceiling, walls, and windows in detail; behind you are rows of empty pews, flanked by huge columns supporting arches that run from the back of the church to the front. Returning your gaze forward, the song reaches its climax and eventual conclusion.

The whole thing is but the length of the song (about 5 minutes), but the stirring performance feels like it was done for you alone, one which feels uniquely immersive thanks to some special technology and a carefully planned and well executed production.

Hallelujah is the kind of experience that is likely to be used for a long while as a demo not only to show the benefits of volumetric capture, but also to stir the imaginations of other creators working in the medium of VR film. Sadly, there’s no word yet as to when (or on what platforms) the experience will launch to the public.