How Sound Reality Affects VR
When VR started its upward trend and popularity a few years back, the initial reaction of those who got to and started using it was the awesomeness of the alternate immersive reality generated. The 3D graphics all around you in 360º gave one the feeling of actually being in the game or app itself. However, what is not widely known is that scientists have been doing studies and research into this immersion phenomenon and the effects on the sensory faculties of a person. VR is not only sight as there are five normally used senses and in VR, sight, sound, touch and smell could make a big difference in perceiving the simulated environment to be actually real.
Many developers have started working on these sensory attributes of VR but at this state of development, aside from the usual seeing the alternate world, if one could actually feel it and hear it as well, one can then attain a realistic "presence" in the environment, the feeling of actually and realistically being there. To attain this, one will need "immersion" which means being totally cut off from your real 3D surrounding (at the moment) and not be affected or distracted by it. That said, these are the two major requirements for a realistic as can be VR experience.
The sense of touch and smell are currently being continually researched and developed at our current time though devices for smelling in VR have to be studied further for safety and health concerns. Touch and the feeling sensation are already being delivered by VR gloves, vests, suits and the like and the developers are busy at work on these. This therefore brings us to the second most important requirement of VR, hearing. To get a somewhat clear grasp of the importance of sound or hearing in VR, let's take a quick look how it affects us in everyday life.
The ability to hear our environment gives us information of the things around us, the place we are currently in and the time. Sound let's us sense things in ways that mere sight cannot (and of course vice versa), but by combining both sight and sound, we are able to see a clearer picture (so to speak) of the whole thing. Hearing gives us the ability to track objects and their position relative to us even if you cannot see them. You may be in a deserted and quiet street but then you suddenly hear a loud screeching sound followed by a bang. You cannot see anything where you are but the sound came from the left at around an 8:00 position. You swivel towards said location with your view being obstructed by a row of houses. But then you are aware that an accident must have happened on the other street beyond at more or less the direction you are now facing at. That is how we all know how sound affects us.
But then, in nature, sound works with sight. You then hear a soft crackling noise as you see smoke billowing from behind the houses. Smoke means something burning, the vehicle must be on fire so you react to these sensory input and you run towards where you heard the crash to see what is going on and help if you can. This is only one example of sight and sound. As they are both natural instincts a normal person is born with, they are usually used for survival as well. Same quiet street but then you hear a fierce growl that seems to get louder and louder though you still cannot see anything. Then you see a huge ugly angry Belgian Mallinois with fangs wide open, dripping saliva and constantly growling at the farthest end of the street but quickly coming your way. What do you do, of course you run for your life! As Mad Max gets closer his growl naturally gets louder.
Now let's apply this to VR. You are in the midst of the Arizona Sunshine deserted Desert. You hear the growling of a bunch of walking dead getting louder and louder. There are a bunch of large crates stacked up in front of you. You move to the side of them to see what's beyond and lo and behold, the welcoming committee is coming your way (some walking, some crawling and some well, appearing out of nowhere). You then decide to move back to a better position, whip out your twin pistols and start picking them out as they appear through the side where you peeked from. But then, you hear a soft growl from behind as you whirl around and shoot the bugger in the head. Without the realistic sound (from behind) you would have been brain food in a snap.
This sound is called 3D audio. With this you will be able to hear sound coming from all around you. Front, back, left, right, top and below. When an under ground water pipe blows up beneath you, you will hear the water gushing under you. A Cessna flies above you, you hear the "wrooommmmm…" and you look up, and of course a heavy breathing and a Bug Blatter hungry hungry hungry behind you and of course you pretend not to see it rather than becoming it's delicious lunch. 3D audio simulates sound in real life especially when applied to games in VR. Using the normal Binaural or stereo sound like listening to music or watching a movie or show may not be enough to give one the audio cues when in the middle of a simulated environment such as an Arizona apocalyptic
With that, sound is very very important in VR and the ability to hear just like in the real world where you can track the position and distance of objects should they emit a sound in the overall gaming or simulation environment. The ability to zero in on a specific sound amidst all other ambient sounds by placing focus on that sound will give such realism in VR. To this, developers like Oculus are continually working to bring 3D and surround sound by using advanced 3D sound recording methods and implementing them into game design by providing tools and necessary resources that app developers can use for their VR offerings.
Hardware wise, even the simplest and cheapest headphones or ear buds can practically deliver 3D audio though in VR and gaming in general, the better the headphone quality of course the better the performance. There are however audio phones that have 3D audio recording capabilities by having microphones as well as speakers on each headphone pad or ear piece. What is important is that in VR, you should be able to hear sounds as they should be and be able to compute distances and direction by listening to them. Thus, developers have to specifically generate or record these sounds and apply them to VR app or game development that even a child could recognize it.