Spatial Audio takes two forms - binaural and object-based (Speakers). Binaural sound is used in headphones and is what we used for this project. Spatial audio is designed to be similar to what we hear in real life, it has the potential to immerse the listener more fully than standard audio. Binaural techniques can be used to create a richer sense of space in composed sound, giving a more exciting and immersive listening experience. Binaural technology can bring the experience of 3D sound to listeners’ headphones. These techniques can be incredibly effective storytelling tools to pair with visuals of the project. For Rising, the plan was to use Immersive Audio to create an Intimate Experience for the listener in the public domain. Our aim was to create a safe listening environment during covid without creating any sound pollution by allowing people to tune in on their devices and thus use their personal headphones. Binaural sound mixing created a truly unique sonic experience for the audience, utilising a 3D soundscape to enhance the audience’s immersion. The real-time binaural audio allowed each listener to engage with the score and sound effects individually, which resulted in a more personal experience, compared to the traditional audio setup for an event.
What Is It?
The Digital Companion is a user friendly web based app that allows the audience to listen to an audio piece that is synchronized with the visual installation and can be accessed througha QR code / symbol at the live event. As we are living through a pandemic, the Digital Companion is COVID-19 friendly as each user will access the web app via their phone and headphones. As it was a binaural experience, the system produced no sound pollution for the location of the event. It was accessible via a private link for anyone who booked tickets for the event, and was easily accessible for all age groups once they had a smartphone and headphones / earphones. It was more environmentally friendly than a traditional audio setup for an event as the system was hosted via the internet. The hardware that was used to trigger the audio via the web app was powered by a large battery array that was charged using a zero emission power source to ensure it is as eco friendly as possible. As the Digital Companion is a web app, it allowed for ease of access compared to a traditional mobile app which would need to be downloaded via the app store. This accessibility allowed less tech savvy users to have access without any difficulty.
The digital companions function was to host the audio piece for Rising, along with some visuals that synced with the show. It enabled the public to listen to the synchronised audio for the visual installation, with sound design from Folding Waves in collaboration with Brokentalkers and Algorithm for Dublin Theatre Festival. - Daniel Staines, Creative Director
Technical Information The Digital Companion started as an exploration into creating a technology that could deliver live audio to large numbers of phones synchronously. The audio had to be high quality, capable of binaural effects and very low latency in order to provide the best experience for Rising. This is because audio delivery would be time critical with the accompanying light show. This posed a number of challenges that were addressed by implementing the application through a web browser. This allowed the application to take advantage of website technologies which were then pushed further to address all our goals. This included the development of a unique peer-to-peer network architecture that greatly improved the live audio synchronization. One of the greatest advantages of this approach was that users are not required to download the application and can simply visit a url.
Rising DC Visuals
The visual aspect of the Digital Companion was divided into two sections:
Waiting Room The waiting room part of the Digital Companion was created for users at the event to explain the key information for the event such as COVID-19 information, how the binaural audio would work via their devices and other important information related to the event. A visual identity which used iconography was created to make the information accessible to all, along with a video of water rising which also acted as the countdown to the show. These visuals were inspired by the original artwork for the show which included rising water, textured layers of color and weather inspired visuals such as heat maps.
During Performance Visuals During the show, the Digital Companion was also used to show visuals that were in sync with the binaural audio. The visuals included stock footage of old advertising posters from the start of capitalism, moving forward to modern day videos of Black Friday, excess and finally the natural disasters that have come to define the 21st century.
