I've attended quite a couple of Meetups on VR/AR and 360 Videos around Munich lately and most of the times the magic word "Immersion" comes up at some point. Surprisingly enough however, a lot of content creators still treat audio as an afterthought and completely neglect it's important role for delivering a truly immersive VR experience.
With this short blog I'd like to raise awareness why good audio is especially important in VR.
Immersion into virtual reality is a perception of being physically present in a non-physical world. The perception is created by surrounding the user of the VR system in images, sound or other stimuli that provide an engrossing total environment -- Source Wikipedia
I'm actually not really a big fan of the word immersion. I prefer the good old
Suspension Of Disbelief [...] has been defined as a willingness to suspend one's critical faculties and believe the unbelievable; sacrifice of realism and logic for the sake of enjoyment. -- Source Wikipedia
VR, more than any other type of media, is about completely diving into and experiencing another world. VR itself contains the term "Reality". It's all about making these virtual worlds as believable as possible. Only then there's the chance to reach total immersion or complete suspension of disbelief.
In order to make the VR user reach this state there's a couple of methods/techniques available. Lets focus on and break down the sensory aspects of VR.
Visual: We perceive a great deal about the world around us through our eyes. In VR, through wearing the headset and by that completely replacing the visual input with a digital feed of images, the user steps into the virtual world. Head tracking allows us to turn our head and take a look around and our eyes allow us to measure the distance of objects and make assumptions about them. But, only if it's in our field of view which only covers about 180°.
Touch: This is still one the more or less unsolved things in VR and there's a couple of interesting projects. Touch allows us to experience materials, weight, heat and cold, force among other things.
Smell: We're not there yet AFAIK :].
Hearing: Like with our eyes we get a lot of information about the world through our ears. Our hearing allows us to make assumptions about the room/environment we're in and about the materials it consists of, it's size, and whether it's hostile or friendly. Through our ears we can also notice things that are happening outside of our field of view, behind us, in the next room or outside. Our ears allow us to locate objects which are elevated or below us. This can be employed to make people aware of events in a VR experience that are happening elsewhere. Hearing also lets us make assumptions whether or not we should consider something harmful, dangerous, scary or even funny. Try watching a horror game trailer, then watch it again with the sound turned of. The list goes on.
With this in mind, it should be obvious that in order to make a VR experience as immersive as possible, we have to treat the auditory aspect of the experience with at least the same importance as the visuals.
One of the arguments (or excuses really) why some content creators think that it might still be okay to neglect audio is:
Nobody will notice anyway.
Thinking like this is the wrong way to approach this medium in my opinion. Our brains react very sensible if things don't behave like we are used to (and we can and should even take advantage of that). For instance, if a VR experience doesn't scale the headset position in relation to the person wearing it, people will notice that they seem to appear smaller/bigger. This breaks the immersion. Getting positional sound and all it's aspects like falloff, direction, elevation etc. right, is more important than ever if you want to make the Virtual Reality believable.
If Total Immersion is your goal, and you treat audio, which is such an important sensory aspect of how we perceive the world, as an afterthought, you're effectively loosing the game! If you really want to deliver great immersive VR experiences that people want to return to, you have to make sure to score in every aspect that's necessary, including audio.
If you have questions about or comments on the topic feel free to drop me a mail via gameaudio[at]m-klier.de.
I'm always looking to collaborate on exciting projects!