Just Like the Simulations – Substitution Reality

It\’s easy to imagine VR as a psychedelic, surreal experience (a bit like being inside of a Windows XP computer); but modern VR is quite clean, immersive and down-right fun. However, we’re far off from a deep and complete dive into the digital world, one of the main reasons being that we will always be acutely aware that we are in VR – at least in its current state.

Standard VR utilises two of the main senses: sight and hearing. Substitutional Reality aims to introduce more of the human senses, such as smell, haptic feedback (sense of touch/resistance), thermoception (ability to sense heat and cold), equilibrioception (the sense of balance and motion change) and proprioception (the sense of space). The theory is, by having dedicated VR laboratories that can simulate these senses, as well as custom controllers, we can create tangibility to VR experiences, to create more immersive and more applicable scenarios (particularly for training purposes).

Firefighter Training

Learn as much theory as you want, but firefighting requires you to, eventually, fight a fire. In standard, practical training, structures are built or acquired, set on fire and students enter the building to extinguish it. There are three main problems with this: the cost of the building; the frequency at which this type of training can happen; and the danger it puts students in, before they are fully trained.

Substitutional reality can provide a solution to this. By creating custom VR controllers designed as fire-hoses, cutters, spreaders, etc., groups of trainees can fight a virtual fire. Heaters can even be used to trigger thermoception. The beauty of this is that common, hard to replicate scenarios can be simulated (i.e. people crying, wooden frames falling, etc.). Also, since there is no burning building to acquire, this training can happen as often as needed, whenever it’s needed.

\”But Bobby; fighting a fire in a virtual world is completely different to real life. There’s no cost of failure in a simulated environment.\”

Yep, I agree. There’s a fundamental difference between being in the real situation and a simulated one. The point of substitutional reality in this case would be to give trainees a better baseline level of experience, before the real thing.

Surgery

The July effect is when there is a noticeably increased risk of medical/surgical errors, as med-school graduates start their residencies. Substitutional reality can reduce the increased risk by giving students unlimited experience with the functioning human anatomy, beyond cadaver use. SR would utilise scalpel, forceps, etc. shaped controllers containing motors designed to give haptic feedback to any given pressure.

However, the biggest benefit of using Substitutional Reality in surgical use is that we can promote mistake driven learning. Cut too deep? No problem, it’s not real. The lack of a time-limit or danger in a simulated environment would allow doctors to consider the route they should take in a relaxed setting and become familiar with the process, which should hopefully translate into a real-life scenario. However, it’s important to note that real surgery isn’t as consistent as SR would like it to be. Different bodies will have organs is slightly different locations; also, people with prior surgeries will most likely have adhesions that will make the process differ.

Substitutional reality does have a place in this world, and it becomes clearer with every new advancement. The problem is that it requires investment and official recognition. It’s fine to say we can use SR for Doctor and Firefighter training, but unless it’s recognised as an effective enough tool by official bodies, it will just stay as a novelty – a proof of concept. However, if successful, SR can change the way we advance this world – if every step is run through a hundred times, every real step we make will be perfect; just like the simulations.

If you\’re looking to hire skilled VR engineers into your team, or if you\’re an engineer looking for a new opportunity, please reach out to me to discuss how I can support you.

Additional Reading

I recommend “Substitutional Reality: Using the Physical Environment to Design Virtual Reality Experiences” by Adelberto Simeone, Eduardo Velloso and Hans Gellersen. It delves deep into the mechanics of substitutional reality, with some great research into how far we can take the experience before immersion becomes too disturbed. The first image is Figure 1 of this paper.

I also recommend “Smart Substitutional Reality: Integrating the Smart Home into Virtual Reality” by Benjamin Eckstein, Eva Krapp, Anne Elsässer and Birgit Lugrin.