How often do you stop and perceive your surroundings with your ears?

As American sound designer Randy Thom explains:

“We are fundamentally emotional creatures who struggle to be rational, and sound plays the role of the puppeteer, pulling our strings more deeply and compellingly than perhaps any other sense”. – Randy Thom

We’re fascinated by the increasing role that immersive audio experiences like DARKFIELD and eyes-closed experiences like Dreamachine and The Hum are playing in increasing not only our immersion in a narrative, but also in increasing our sense of wellbeing. There’s also a growing conversation around how experiences that don’t rely on seeing can not only be more inclusive for the visually impaired, but for the wider world.

So for this WXO Campfire, sound designer Dora Filipović shares her research into the connection between sensory perception, artistic expression, and the experience of blindness. Filipović believes in the meditative qualities of perceiving surroundings through heightened auditory senses. 

In her short film Return To Impression, the phenomenological experience of living with sight impairment can be experienced through ‘immersive’ sound and vision via a VR headset. Filipović’s work is a springboard into thinking about the ways we might use sound to enhance our experiences – not just for those living with visual impairment, but for everyone.

The Connection Between Sound, Light, & Blindness

According to 2020 data from the Lancet Global Health Commission on Global Eye Health, 596 million people had distance vision impairment, 43 million were blind, and 510 million people had uncorrected near vision impairment. 

Family, friends, teachers, and society need to understand the experience of people with these conditions to facilitate care, communication, and counter-discrimination. But the current methods of simulating sight loss involve using blindfolds and goggles that obstruct vision. They don’t accurately replicate the experience of living with a visual impairment, which is most often compensated for through a heightened awareness of the auditory environment.

Inspired by this research, Filipović set about to better understand how blindness affects painters. An investigation into the perspectives of blind and visually impaired individuals in Serbia revealed a transformative connection between sound and light in the absence of visual stimuli. Every experience begins with consciousness and awareness – and blindness can inform an equally relevant experience of the world.

“If we can abandon the notion that blindness can only diminish, damage, or destroy identity, and adopt instead the idea that experience of blindness… can in fact shape and inform other facets of personality and personal history, we will move toward a more genuinely inclusive society.” – Georgina Kleege

The Subjective Experience Of Blindness

To take people to a surreal state of imagination driven by sound, Filipović developed a film designed to be experienced using a virtual reality headset with immersive sound and vision. The aim was to depict the phenomenological experience of living with a sight impairment, involving accentuated attention to the auditory world. 

The development of the piece was informed by ethnographic and historical research into the effects that blindness and sight impairment have on “visual’ artists”, and how in order to continue and sustain their artistic practice, they must rely on memories of past experiences and more acute perception of environment sound associated with the subjects and surrounding they wish to portray, ultimately using imagination and instinct rather than merely visual observation. 

The questions Filipović sought to answer through the project were:

• What is the subjective experience of sight loss and how can we portray it?
  
• How effectively can we depict this experience using an audiovisual and immersive platform?
  
• To what extent does the sound of an environment or subject awaken emotion and memory through visual expression?
  
• How can we measure and enhance empathy for blindness through the “embodiment” offered by an immersive audio-visual experience?

The narrative of the film was that an artist, Filip, who is losing his sight, is visited by the muse Vera. She is a personified force, a metaphor, and the source of inspiration for the artist. Afraid that Filip will give up painting completely, Vera takes the artist on an inspirational journey to help him paint the scene by listening to the sounds which she experiences. 

Through the subjective lens of 360 cinematography, and the person-centric interactive experience of fully immersive sound design, the intention is that each audience member experiences Filip’s journey for themselves, whatever their level or lack of visual sensory capability, or indeed their affinity towards enhanced auditory perception.

The Power Of VR To Generate Empathy

15 sighted people and 15 blind and visually impaired people were invited to experience Filipović’s film as part of audience research/ They were given the choice of experiencing the film with full visual or no visual input, before they were interviewed afterwards.

These answers revealed that VR provided an immersive experience that allowed the audience to explore visual impairment through sound, giving full freedom of orientation to the audience and demonstrating that the memory of experience could be evoked through sound alone. According to one male participant, aged 79, who only had headphones:

“I could almost feel the wind on my face when pebbles were being crunched as people walked across the beach, and the waves were crashing towards me.”

Participants also empathised not only with feelings of frustration or disconnection, but also with Filip’s determination to pursue his practice despite adversity. According to one female participant, aged 65, who had both headphones and a headset:

“I absolutely identified with him in terms of him saying about wanting to get on with his life. I think that’s a really key message, that you become your eyes for a while, and actually you have more than your eyes.”

This reminds us of other examples of how VR is being used to create experiences that generate empathy and allow people to engage with difficult topics, such as:

• The Severance Theory: Welcome To Respite, a VR psychological thriller that “explores the parts of our minds that both help us and haunt us through the lens of a mental illness called Dissociative Identity Disorder”
  
• Goliath: Playing with Reality, a VR film that “explores the limits of reality and a true story of so-called ‘schizophrenia’ and the power of gaming communities” and won the Grand Jury Prize for Best VR Immersive Work at the 78th Venice International Film Festival
  
• 1,000 Cut Journey, a game on the Meta Quest platform that aims to help users understand the emotional and physical toll of racism

“My aim was to inspire people to think about how to incorporate sound into their work. If you become aware of how a certain person is living, it’s not just creating empathy, but also helps you think about how to incorporate it into the world.” – Dora Filipovic

The Next Generation Of Immersive Audio Technology

There are already examples of immersive audio technology on the market, both custom and more widely accessible. Think:

• Jennifer Crook’s Dreamachine, a multisensory experience with 360-degree audio
• Gen Cleary’s The Hum, a 360 spatial vibro-musical-immersive experience.
• Apple’s VisionPro headset
• Holoplot sound, which is already being used at Sphere, Lightroom, Illuminarium, and Horizons of Khufu

Holoplot may be at the cutting edge of the genre, but the expense means it may still be inaccessible for many creatives to add real value. However, technology doesn’t have to be cutting edge to add value – it just has to support the creative idea.

“Originally films didn’t have sound departments, or surround sound. Changing the technology is secondary to what you can say through artistic practice. How can you show your creativity through technology, and not let it limit you?” Dora Filipovic

As Layered Reality’s Andrew McGuinness says, technology is only one of the three layers they turn to to realise the story, alongside theatre and the physical senses. And AR expert Rob Morgan says that to truly immerse players in your narrative, you don’t augment the tech – you augment the story in their mind.

The WXO Take-Out

“Sound is what gets you to that next dimension. A creative director I once knew said, ‘Ears do not have lids.’ You hear with your entire being.” – Lou Murray

We often talk about how to use technology to further the story in an experience – but Filipovic’s work explore the idea that we should give equal weight to how to use the senses to further the story, too – as well as create truly inclusive, shared experiences.

So next time you’re designing an experience, ask yourself:

1. How might you use sound to evoke memories and feelings? 
2. How can you create a bigger impact by accentuating one sense – or removing one or more senses?
3. How might you create experiences for visually impaired or blind people?

Want to come to live Campfires and join fellow expert experience creators from 39+ different countries as we lead the Experience Revolution forward? Apply to join the WXO today.