A major French project involving Brain-Computer Interfaces applied to video games (ANR-CONTINT)

17

01

2013

 

OpenViBE2 (2009-2013) is a collaborative research project, supported by finance from the ANR, that is based on the potential of the technologies known as “brain-computer interfaces”(ico) in the field of video games.

This is a project that has brought together the scientific expertise needed through a multi-disciplinary consortium consisting of nine partners - the university laboratories who pioneered the field (Inria, Inserm, CEA, GIPSA-Lab), well-known video-game manufacturers (UBISOFT, BLACKSHEEP STUDIO, KYLOTONN GAMES) and specialists in usage and transfer (LUTIN, CLARTE).
After three years of work and achieving numerous scientific advances associated with the development of innovative industry prototypes, OpenViBE2 has made it possible to have greater control over the future of such technologies on the French market as well as internationally.

A major French Project involving Brain-Computer Interfaces applied to Video Games

Acting on thought thanks to brain-computer interfaces

A brain-computer interface enables users to send commands to a computer purely by using their mental activity. BCIs use electroencephalographic (EEG) devices based on electrodes that are placed on the surface of the head, and that record electrical signals corresponding to exchanges of electric current between the neurones. The electrical activity produced by the brain is then analysed in real time and translated into a command sent to the computer or any other automated system thus making it possible to move a cursor to the right or left, simply by mentally imaging the movements a hand would make.

Initiated in 2005 by Inria and Inserm, the first OpenViBE (2005-2009) project, which was also supported by the ANR, was completed in 2009 with the creation of open-source software that made it possible to easily design, develop and test brain-computer interfaces (http://openvibe.inria.fr). Today, this is recognised software that is used throughout the world. The OpenViBE2 software is a sequel to this initial project and is aimed at exploring these technologies more thoroughly by exploring them in a particularly complex applicational context - that of video games. This is a relatively new field of application for BCIs. But the advent of low-cost EEG helmets has opened the way to such applications.

The purpose of OpenViBE2 is thus to improve the current abilities of BCIs and test their use and their potential in the field of video games.

Video games: a growing market

As a development of the “gesture-controlled” command (joystick, mouse, gamepad), the “mental” command is a new and very promising route in the field of video-gaming. Since the early 2000s, scientists have been tacking the scientific issue of how to incorporate BCI technologies to enable them to interact with virtual environments.

In this context, the aim of the OpenViBE2 project is to identify and use the gamer’s mental state and brain responses to interact with the game and/or adapt the actual content of the video game accordingly. A very original approach suggested in the project was to consider BCIs not as a substitution technique for the classic interfaces (joystick, mouse, gamepad) but rather to consider using the BCIs as a new way of playing that complements the traditional techniques. In this way, the gamer can continue to use a joystick while at the same time or at certain key moments in the game, he/she can also mobilise his/her brain activity for playing.

Advances in the OpenViBE2 PROJECT: promising innovations

Scientific advances

Whether in respect of diseases and neurological syndromes, the world of entertainment or daily life, OpenViBE2 opens the way to innovative technologies and major interests for human beings. During the course of the OpenViBE2 project, major scientific advances have been obtained in these three fields:

In Neuroscience: identification of new mental activities linked to the attention processes.

In the first leg of the OpenViBE project, the Inserm team known as “Mental Dynamics and Cognition (Dycog)” at the Lyon Neuroscience Research Centre (Inserm/CNRS) participated in the development of software capable of “writing through thought”, thus facilitating communication by people suffering from a motor handicap. OpenViBE2 offers new advances in the field of neurosciences that makes it possible to use interfaces brain-computer for therapeutic purposes in order to improve certain neurological disorders such as attention deficit disorders.

Thanks to the device, researchers were able to analyse:

  • attention paid to the outside world by measuring in real time and selectively the level of engagement of the cerebral network responsible for seeking specific information in a visual scene.
  • attention paid by the user to an internal representation, c'est-à-dire i.e. the level of engagement of the cerebral network responsible for maintaining mental representation.
  • The level of distraction of a person by determining the real time in which a person is distracted and for how long.

The principle of virtual reality, used mainly in the serious game ADHD developed thanks to OpenViBE2, enabled researchers to obtain results with respect to attention deficit disorder. The virtual environment is similar to the real environment and the user has to concentrate in order to perform a task that he/she knows from the real world. This virtual training employs the process known as NeuroFeedback in which the user is required to self-regulate his/her mental activity.

The serious game, which is associated with augmented reality, has a positive retraining effect that persists beyond the training session and into the real environment”, stresses Jean Philippe Lachaux, Director of Research at Inserm.

Progress is still necessary with respect to the device so that it can be accessible to all”.

In processing signals

The CEA and GIPSA-Lab researchers have found new techniques to better filter and interpret electrical signals from the brain. These make it possible in particular to eliminate noise interference (especially when it is linked to muscle activities such as facial muscles, blinking or jaw-clenching), and extracting the relevant brain activity as accurately and in as focused a manner as possible. These techniques are very useful in a gaming context, in which gamers may be very mobile.

In virtual reality and man-machine interaction

OpenViBE2 has enable the INRIA researchers to invent new concepts to interact with video games in the most original and effective way and suggest new world firsts:

  • The brain-computer “multi-player” interface: the INRIA researchers designed the very first collaborative or competitive gaming application in which the brain activities of two gamers were analysed simultaneously. Both players could play on the same side or in opposition to each other, in a simplified video football game.
  • The natural incorporation of BCI into the virtual world: researchers studied how to best incorporate the stimulation necessary for certain brain-computer interfaces based on mental responses, by incorporating visual stimuli into the virtual world. For example, some BCIs “visual flashes” that are recognisable in the user’s brain. These can be incorporated into a video game, for example in the form of butterflies that beat their wings at different frequencies.
  • Automatic adaptation of the virtual world to the gamer’s mental state: Finally, researchers have offered radically new approaches in which elements of the virtual environment have been automatically modified depending on the player’s mental state. In a virtual maze game, guides are thus activated automatically if the user presents with too great a mental burden. This work has benefited internationally through the award of several science prizes (Best Paper Award, Eurohaptics 2012, BCI Award 2012 nominee).

The gaming room of the future

The OpenViBE2 project has made it possible to develop numerous proofs of the academic concept and commercial video game prototypes all of them activated by the brain.

Video game manufacturers have worked directly with the Inria university laboratories, as well as with Inserm, CEA and the GIPSA-Lab, to develop video games based on brain activity. This collaboration has made it possible to better define scientific research initiatives throughout the project and bring all the research together into solutions that are best suited to the technological constraints of the field of application.

During the course of the project, the partners introduced a huge campaign of experiments involving “brain-computer interfaces and video games” run by CHART at the Cité des Sciences in the course of which nearly 400 testers tested the prototypes. The results of these experiments made it possible to highlight the attractions of these technologies to a wider public, and provide the consortium of partners with a large amount of user feedback.

A standardisation project was also run by UBISOFT to define the “ideal” helmet best suited to the video-games market.

Know more

Press Contacts

Burson-Marsteller I&e
Lise Gallard / Alice Dalla Costa
Tel.: 01 56 03 12 87 - 01 56 03 12 26
lise.gallard@bm.com / alice.dalla-costa@bm.com

Inria
Muriel Droin
Tel.: 01 39 63 57 29
Muriel.droin@inria.fr

View Edit Create here