Thibault Paquin interviews the editor of UXmmersive and founder of Onyo, Charlotte-Amélie Veaux, about the importance of sound in creating an immersive experience, even without visuals.
On a recent visit to Paris, I had the chance to experience the new talk-of-the-town, the gorgeous Hôtel de la Marine, recently opened following a 6-year transformation from the Ministry of the Armies into a sophisticated lifestyle venue on the Place de la Concorde. I was curious about its much-praised 3D sound installation, and I was not disappointed.
Forget about old-school audio systems: this thing knows where you are and talks to you in the softest way. As a result, there’s no need for cluttered rooms with signage and interactive displays. Just the simplicity of beautifully renovated rooms, amazing “dancing mirrors” with smart visual content and your audio companion. And when baroque music started playing as I was walking down the stairs on my way out, I felt like I belonged. I was very close to an immersive experience!
To explore the subject of 3D sound and its role in LBE (location based entertainment) venues in more depth, I talked to Charlotte-Amélie Veaux, the editor of UXmmersive, founder of Onyo and one of the members of the Celebrating Life collective.
Before you continue reading, you may like to experience Onyo. Click on the image below to listen. Our advice: it’s best with headphones on and eyes closed.
Can you tell us about your journey with UXmmersive?
After studying philosophy and geography at La Sorbonne, I started as a consultant in open innovation for a Paris-based firm. This is where I met my partner, Yann, and we decided to go on a round the world trip of immersive experiences. This was in 2019 and, retrospectively, I am glad we did it then. It won’t be possible for a long time!
We wanted to explore innovation, and immersive experiences in particular. We were interested in the subject because it is at the intersection of video games, which Yann is a fan of; innovation, something we had in common; and performing arts, of which I am fond, especially dance. We thought the notion of immersive experience wasn’t addressed much in France yet, only in very specific fields such as museums, marketing and technology, but not in a holistic way. We were convinced there was something to explore.
We left for a year and went to South America, North America, East Asia and East Africa. Our goal was to test a lot of different experiences – more than 150 in total – and interview people such as entrepreneurs, designers and academics to figure out what an immersive experience is, how to create that feeling of immersion, and what the different business models are.
When we came back to France, we started our blog UXmmersive to talk about what we learned and to keep documenting immersive experiences. Then COVID happened, and after looking around we decided it was time to take the leap and create our own immersive experiences. When we started, we had a blank canvas. We asked ourselves what kind of experience we wanted to create, using all the insights and feedback we collected during our trip.
How did you come up with Onyo, a mostly audio-based immersive experience concept?
The most amazing experiences that inspired us were sensory and narrative. We also liked collective experiences.
From our interviews, we also gathered that the economics are difficult and that scalability is key. For example, having actors is great for the experience, but hard for the business model.
We knew we had limited funds and it was our first creation, so we couldn’t start with something too big or too complex. We asked ourselves some questions. How can we create an impact without actors? How can we create something that can be scaled easily and replicated? What immersive lever, technology or other, can we use? And that’s how we picked audio.
Binaural sound is very immersive, a stable technology and not too expensive to produce. We had also tested Darkfield experiences in London and were completely amazed. Even though I’m not into horror experiences, we fell in love with the concept of a 15-minute audio experience in the dark.
For the narrative, we looked around us. It was in the middle of COVID and everybody was talking about climate change. So we wanted to create something for people to escape and explore magical worlds. We came up with a moodboard that pulled us towards the universe of Studio Ghibli: something soft that can talk to children, young adults and corporate people at the same time. We wanted to invite people to wonder and marvel about small things in their everyday lives. For example, in our first creation, The Tree of Light, participants are in a magical forest, but the actual sound recording comes from a forest just outside Paris. This is to show you don’t have to go far to find magic.
Can you tell us more about (3D) sound and immersive experiences?
When we think about immersive, we mostly think about VR. During our trip, we tested a lot of VR experiences but we didn’t find they were the most memorable. They didn’t give us the most goosebumps, the most thrills. On the contrary, we feel sound or touch can do that because they are deeply connected to our emotions. They give us a lot of information for our brain to process.
For example, when you read Harry Potter, you can imagine how Harry, the castle or the magical creatures look down to the smallest details. I’m a Harry Potter fan, but I didn’t like the movies. Because they were not as good as my imagination. With a sound experience, it’s more like a book than a movie. You will hear animals or people, but who knows what they look like? So you can imagine whatever you want. You are more active in a sound experience than in a visual experience, because your imagination has more space. And that’s a first step towards an immersive experience.
But participation is the main characteristic of an immersive experience: how to make the spectator an actor. In teamLab work, people participate. They touch bubbles, water, etc. In Darkfield or other immersive theatre experiences, people are part of the experience.
It is the same with Onyo. We are not telling a story, we are making people live the story. There is no narrator. With 3D sound, you are in the scene. In The Tree of Light, you are participating in a ritual with animals and a shaman-doe.
Has there been much evolution in sound technologies and production?
When we talk about immersive technologies, we think first about VR and AR, which have drawn the most attention, production and financing. Audiovisual projects are mostly visual projects, with a bit of audio! In France for example, there is a strong push for digital, audiovisual and performing arts, but not really for audio only. With the rise of podcasts, things are slowly changing.
I strongly believe that audio-based experiences will grow. Binaural sound is kind of VR for the ears; it really takes you to another world. It’s been around for decades, but has not really found its audience. I think it’s starting now for different reasons. One is widespread use of good-quality headsets. Everyone has them now, especially with people working from home. Another reason is a lot of people are suffering from a screen overload.
Also, the quality of content is improving. Creating a good immersive experience is a combination of good experience design and good writing. You can gather the best technologies you want, but if you have a bad design, the experience will not work. We are now at a time when the whole industry is learning, especially how to address the narrative. Good writing is based on immersive theatre techniques. The Darkfield founders, for example, come from immersive theatre. Like with VR in the past 10 years, there is a maturity required to produce better immersive sound experiences.
How can sound assist in storytelling and creating emotions?
Without sight, audio becomes very important. We were often given the example of a horror movie. If you watch it without sound, it’s not scary anymore, but if you close your eyes and just listen to the sound, you will still be scared. Sounds drive a lot of emotions. And like with other senses, it is very personal. It will bring different emotions to different people.
Immersive experience designers all agree soundscape is key to creating an atmosphere. 3D sound offers new possibilities to make it even more real and to turn the participant into an actor instantly.
There are more and more podcasts produced in 3D. But listeners remain in the audience; they follow a story being unfolded. It’s not immersive. What we do with Onyo is a truly immersive experience, where we use an immersive technology (3D sound), immersive writing (where people are part of the experience) and some set design (tree of light, tent).
Where do you see 3D sound going in the future?
The next big thing is headtracking. When people move their heads, they can hear where the sound comes from. They can get closer to a sound source, for example. It’s then possible to spatialise sound and create new worlds, like a sandbox.
Some startups are already working on headtracking solutions, but it is still quite expensive and not easy to implement. As early as next year, I am sure it will be available for use in immersive experiences.
We wanted to include it for our first Onyo experience but we decided to wait for future experiences, when you should expect them. So stay tuned on Instagram or LinkedIn!
Can you tell us about some of your favorite audio-based immersive experiences?
Darkfield of course, but also some binaural podcasts such as Hasta Dente (in French) produced by France Culture, the concerts in the dark by Nuits Noires, or the very funny phone-based experience Telelibrary.
To find out more about immersive, read up about our ‘What Is Immersive?’ Campfire 21. To become a better experience designer, join the WXO.
