Invented several decades ago, the concepts of virtual reality and augmented reality have received very significant media spotlight in recent years. The main reasons for this are the significant democratization of technologies and the very strong investments made by the GAFAMs on these subjects.

Among the fields of application likely to benefit from these developments is training, a field for which virtual reality or augmented reality offer numerous advantages . Without being exhaustive, the following advantages can be noted:

• the possibility of carrying out experiments without any danger for the learner or for the material. This is the typical case of a very expensive device that a learner would be likely to damage during his training;
• easy role-playing of the learner , including in critical situations, such as the malfunction of equipment that can become dangerous;

• total control of the learning situation and follow-up of the learner. This allows the trainer to have access to a learner’s progress data at any time and, in some cases, to all the actions performed by the learner by virtually “replaying” the training sequence.

The technology is therefore already old and we would be tempted to believe that all the pitfalls are listed and that it is easy to create a virtual reality application, for example in the field of training. Experience shows that it is not that simple and that many risks, moreover very well documented in the scientific literature, are not sufficiently taken into account by developers in the development of applications.

Latency

To illustrate the point about these difficulties encountered by application designers, we will take a few examples.

First, to be reproduced in the virtual world, even the most elementary physical phenomena of the real world must be simulated, whether it is the effect of gravity, contact or collision between objects, the propagation of light… These simulations often result from the resolution of complex equation systems that require computing time from computers, and therefore delays during interaction with the user.

The instantaneity of physical phenomena cannot be reproduced in the virtual and this has consequences on the user’s perception of them. Indeed, the delay also called latency can disturb the natural character of the interaction.

Let’s take a very simple example, the vestibulo-ocular reflex, which links visual perception and the movement of the head picked up by the inner ear: stare at an object and move your head, the vestibulo-ocular reflex allows you to stabilize your gaze on the object. In virtual reality, the slightest latency in the process will be perceived by the user. Perception will be disturbed: objects supposed to be fixed (buildings, doors, walls, etc.) will no longer be so in a virtual environment. In some cases, he will experience what is called simulator sickness (Cybersickness).

Another difficulty faced by those who want to use virtual reality for training purposes: the perception of distances. Scientific experiments have clearly demonstrated that these are very frequently altered in the virtual, sometimes there is underestimation, sometimes overestimation. It is therefore logical that, in an application for which the perception of distance is paramount, such as learning to shoot basketball, a user may have difficulty learning.

Constraints to be aware of

Furthermore, the fact of being immersed in virtual reality disrupts the phenomenon of “accommodation vergence”: this principle of human visual perception consists, when looking at an object, on the one hand in placing to the point (accommodation) on the object according to its distance) and, on the other hand, to orient the eyes to converge on this same object (vergence). This mechanism is very synchronized since the accommodation and the vergence will be carried out according to the distance to the object.

In virtual reality it is very common for accommodation to be performed over a fixed distance while vergence is performed over a 3D distance which will depend on the position of the object. There is therefore a desynchronization of vergence accommodation which requires the user to make a particular effort which, if necessary, can become very inconvenient.

Finally, we learn from childhood to interact with our environment when inspecting and manipulating objects. In a 3D environment, if the interaction devices and the interaction metaphors are not perfectly adapted to the task to be carried out by the user, then the latter will have difficulty in carrying it out. A typical example is the use of a classic mouse, which moves on a plane, to achieve the rotation of an object in space, it is not strictly speaking the ideal means. In a somewhat colorful way, the user would experience a handicap in the virtual compared to what he knows how to do in the real world.

As we have just seen, there are some pitfalls in developing virtual reality applications and too many developers are unaware of these limitations. Nevertheless, it would be too extreme to think that virtual reality does not work. Indeed, subject to proceeding very methodically with their analysis and development, it is quite possible to create applications that work while respecting certain constraints:

• it is necessary to control the complexity of the calculations in order to control the latency;
• it is necessary to study the scientific literature on perceptual inconsistencies in order to avoid putting oneself in such situations;
• concerning the naturalness of the interaction, it is also necessary to study the scientific literature to evaluate, taking into account the specifications of the application, the best interaction metaphors so that the user can be comfortable and effective when performing the task in a virtual environment.

Check learnings

Once the basic precautions have been taken, in a training application, we also need to know if the learner has learned – it is the least we can expect – and if a transfer of learning has taken place. between the virtual device and the real device. This question of transfer is fundamental since it is all the same the primary goal of the operation.

In this case, rather than simply hoping that the transfer takes place, it may be wise to verify it experimentally, that is to say taking real users and having them pass validation tests.

An example of practice is to take a subset of users who learn in a classical way (control group) and another subset who learn by the virtual reality application. Appropriate analyzes on the two cohorts must make it possible to evaluate understanding, acquisition of know-how and/or procedure, memorization after a certain time in order to validate the quality of learning in the virtual compared to the real. Further analyzes are necessary when comparison with reality is impossible (case of rare or dangerous events for the learner and/or the equipment).

In conclusion, the general picture is neither totally white (in virtual reality everything works wonderfully) nor totally black (nothing works). We are clearly in a middle ground. Subject to mastering the constraints of creating applications related to the intrinsic limits of a user’s perception, it is possible to create training applications that work perfectly.

Recently a new terminology has appeared: metaverses , which are increasingly present in the media and in industrial marketing. However, the concept and the scientific works on the subject date from the beginning of the 90s and the technical heart of metaverses is built on very old technological elements such as virtual reality, virtual humans, networks… So it’s a very nice marketing coup of the company that carried this buzz.

Of course, the fact of pointing the spotlight on this subject will lead several companies to invest and several laboratories to take the subject head on, it is therefore a safe bet that in the months and years to come this concentration of means will produce effects, therefore new developments and innovations from which training will benefit. It is also not useless to ask the question of the risks on the evolution of society that metaverses make us fear: is this really the society we want? Is the game worth the candle?

Author Bio: Bruno Arnaldi is a Computer Science Professor at INSA Rennes