GUEST:
Almost 90 years ago, in October 1927, the art of storytelling took a dramatic turn when the first “talking picture” or “sound film” was released to audiences around the world. The Jazz Singer was a smash hit, earning more than $2.6 million as it captivated audiences with its Vitaphone technology, the era’s leading sound-on-disc system that would forever change the standard for in-theater experiences. Within just a year, filmmakers encouraged by the success of The Jazz Singer were already developing ways to advance in-theater audio beyond the use of discs and turntables and began migrating to sound-on-film. Regardless, the new multi-media era had been launched. The introduction of sound on top of moving pictures took people to a new dimension.
Fast-forward to today and the art of storytelling in our modern, more virtual world is once again about to be disrupted by another seminal advancement in sound. It’s the use of 3D audio to immerse people more deeply into virtual environments, taking them to a reality that’s, well, much more real.
Binaural 3D audio is inherently more authentic to our ears than two-dimensional stereo. It is sound that’s designed to replicate the way we hear spatially, leveraging how humans consume auditory information in our natural environment. 3D audio engages the listener by offering a spatial bearing that enables them to sense where they are relative to the noises around them. In a 3D soundscape, the origins of sounds can perceptibly move about the listener, locating the listener as if they were standing in a real life environment.
By inserting 3D audio, new spatial information is introduced to the virtual experience, enabling audiences to sense things happening behind them, or elsewhere in their virtual environment, completely independent of their eyes. Just imagine yourself at a museum, standing in front of a massive painting that – despite being a two dimensional work of art – has amazing depth and allure. Visually, it draws you in and creates a sense of rapture. Now fold 3D audio into the experience with carefully crafted sound that wraps itself around you and pulls you closer. Characters in the painting sneak up from behind and encourage you to look deeper. Meanwhile, others call for your attention on the right, and then on the left. Suddenly, you’re a part of what’s happening on the seemingly “flat” canvas. Manipulating this type of audible sensory perception has the potential to completely reshape the entire virtual experience.
While the creative industry has been exploring 3D audio for some time, these efforts have been difficult, expensive and time-consuming to engineer since its production requires a mannequin head equipped with microphones shaped like the human ear. Not exactly the type of stuff you have laying around in the garage.
However, a group of five companies called The BINCI Consortium – short for “binaural tools for the creative industries” – are working collaboratively to develop an integrated software and hardware solution that can be used by professional audio content creators and artists to ease the production, post-production and distribution of 3D audio content. As an active member of The BINCI consortium, I share the organization’s vision that everybody will soon be able to create and listen to binaural audio with off-the- shelf devices and headphones. The Consortium, which includes my company, Antenna International, as well as Eurecat, HEAD acoustics GmbH, 3D Sound Labs and Voodoopop, aims to develop a solution that can support a variety of professional applications in the creative industries, such as music, video games, virtual reality and augmented reality. The new tools will cut production costs tremendously and therefore revolutionize the industry – as well as all virtual experiences.
Virtual reality, augmented reality and mixed reality experiences as we know them today are still “flat” unless the audio sound doesn’t create a similar three-dimensional reality as the visual effects do. Reflecting back to how Vitaphone technology changed the in-theater experience nearly a century ago, today’s virtual experiences haven’t yet met the modern equivalent of The Jazz Singer – but it’s coming soon.
BINCI’s experimental production pilots are scheduled to be carried out over the next year in cultural and heritage sites that include Fondació Joan Miró (Barcelona, Spain), Opéra Garnier (Paris, France), and Alte Pinakothek BStGS (Munich, Germany). These pilots, also known as “The BINCI Project“, will offer visitors the world‘s first encounter with 3D audio-guide productions and usher in a new era of immersive storytelling.
In the meantime, others in the VR, music, film and gaming industries are also attempting to drive 3D audio forward. For example, Microsoft’s new Xbox One X supports 3D audio content – and the headphone manufacturer Plantronics has developed 3D audio gaming headsets. And San Diego-based Comhear Inc., has recently developed a sound projection system that can deliver 3D audio without the use of headphones.
In less than five years, 3D spatial audio is expected to revolutionize our standard for multimedia listening. Similar to how high-definition television has enhanced the everyday viewing experience, binaural 3D sound is expected to reshape our listening experience and redefine the production of music, movies, radio, and television programming – and yes, VR, AR and mixed reality content as well.
There’s currently no blueprint for piecing together the storytelling that’s best suited for this new type of virtual medium. As far as content goes, there is consensus about only one thing: in a virtual world, it’s all about “storydoing”, not storytelling.
Eva Wesemann is the Director of Creative Strategy for Antenna International, a provider of technology, content, and managed services to the world’s artistic, historic, and cultural institutions. She is also an active member of the BINCI Consortium.
The PC Gaming channel is presented by Intel®‘s Game Dev program.
x
Thanks for your interest in the PC Gaming channel
Select the products you would like to subscribe to:
Country required
*By checking this box, you are confirming you are an adult 18 years or older and you agree to Intel contacting you with marketing-related emails or by telephone. You may unsubscribe at any time. Intel’s web sites and communications are subject to our Privacy Notice and Terms of Use.
Source: Why 3D audio is the next big step for virtual reality
