The Swiss lakeside town of Montreux will once again host its famous Jazz festival. Over the past 50 years, the festival has seen some of the greatest acts in music history, including Nina Simone, Ray Charles, Miles Davis, Prince and David Bowie, among others.
For this year’s festival, audioborn has developed a project allowing the festival to be captured through 360° / 3D video and sound recording technology for the first time. This content can later be rendered and used in virtual reality applications, permitting users to experience the selected acts in real-life quality.
Commissioned by the Metamedia Center of the University École polytechnique fédérale de Lausanne (EPFL) together with the Montreux Jazz Festival, project lead and official technology partner audioborn is partnering with PRG Lab for the video expertise. audioborn is responsible for capturing and reproducing all concerts in 3D audio with its software real-time processor called Auratorium. The software system is designed for 3D audio and virtual reality applications. It uses a ray-tracing approach to calculate physically correct and 100% natural sound.
PRG Lab, a division of PRG XL Video, based in Germany, develops customized interactive and leading software solutions for the event and exhibition market. PRG Lab will be supporting the project with a 360° / 3D Nokia OZO camera system to capture a 360° spherical video of chosen acts at the festival.
The cooperation between audioborn and PRG Lab will be one of the first commercial uses of the OZO system in a production of this size. In addition to the standard multitrack recordings, audioborn will capture the sound in the form of a 3D recording, using the Sennheiser VR microphone prototype, and be able to reproduce highly realistic 3D audio for the recordings.
“The methods we use to capture sound – which is the most important part of Montreux’s Jazz Festival – will be pioneering for 3D audio sound recording as well as reproduction. Using our 3D software audio processor Auratorium, we will be able to achieve the highest possible degree of immersion for the jazz archive.” says Dr. Dirk Schröder, CEO of audioborn.
“We will have to determine how each instrument and sound will react depending on the position of the camera in the room. When sound and video are combined for VR playback, the virtual user should feel as if he or she is actually at the festival in person. Our goal is a total immersive playback experience in sound and vision. We are proud to work closely together with PRG Lab as our professional partner for 3D video capturing.”
“This project is outstanding in its dimensions. We are going to record a huge amount of data during the 16 festival days with the OZO system. This will be the most demanding challenge we have to face during the project.” commented Michael Ochs of PRG Lab. “This is very exciting, state-of-the-art technology being used at the Montreux Jazz Festival. It is the first project of its kind and we can’t draw from previous experiences. ”
One of the challenges of this project is the large amount of data collected through 3D recording. A concert of three hours would create approximately 2 TB of compressed data. Uncompressed it multiplies later in postproduction by factor 12.
PRG Lab is continuously expanding its 3D capabilities and will be using OZO production computers as well as the audioborn Auratorium software to create immersive experiences. The audio mixes can be rendered for multichannel 3D audio systems or binaural headphone reproduction.