Title : Realistic sound source simulation in multimodal virtual environments: Testing the ventriloquism effect with surround systems

Project Lead : Trk goston From : Doctoral School of Psychology, Etvs Lornd University; Institute of Cognitive Neuroscience and Psychology, RCNS HAS, Research Group for Developmental Psychophysiology (None)

Dates : from -- to 2014-10-06 12:04:26

Description :

Motivation and objectives :
Motivation: Our research group is interested in the cognitive dynamics of audiovisual integration. We have recently extended our research to VR environments. We carried out a succesful experiment using the VisionAir infrastructure and showed that audiovisual ventriloquism may work in virtual reality too. Based on our earlier experiment, we would like to further elaborate our knowledge on how ventriloquism can be used in practical applications. Mostly we are interested in how this effect can contribute to design of audiovisual warning signals. Objectives: Virtual reality environments nowadays are highly realistic, but research has mainly focused on visualization. User experience could be enhanced by using multimodal environments which, in addition to visual information, contain audio information as well. This way we can test such technological solutions that would be otherwise dangerous or expensive. The CAVE system provides an excellent opportunity to test novel approaches in audiovisual warning system design. Although several studies focused on how audiovisual signals can enhance reaction times to potential dangers, to our knowledge none of them studied how the signal and the signalled object can interact and integrate to a common mulitmodal object. We seek to find how and when this integration occurs and how it can effect the speed and adequacy of responses. Audiovisual integration is believed to work on a perceptual level, thus phenomena such as ventriloquism could give us better insights to what extent virtual reality is reality, and to what extent it is virtual.

Teams :
The Research Group of Developmental Psychophysiology was founded by Valria Cspe in January 2000. Our main research method is using EEG the measure event-related brain potentials (ERP), i.e. the time-locked, synchronized electrical activity of the human brain, elicited by various stimuli. Our primary research topics include the processing of auditory events, language processing, reading and number processing. These topics are studied in both adults and infants, and several other methods of cognitive psychophysiology, experimental and cognitive psychology and cognitive neuropsychology are applied.

Dates :
starting date : 07 September, 2014
ending date : 20 September, 2014

Facilities descriptions :
http://visionair-browser.g-scop.grenoble-inp.fr/visionair/Browser/Catalogs/CRVM.FR.html

Recordings & Results :
We designed an experiment where we tested how effective spatial sounds are in signaling dangers and facilitating reactions. For this we divided the project into four parts. In the first part we collaboratively planned the experimental set up, the possible ways to measure facilitation effect. In the second part, Agoston Torok visited the CRVM lab and they jointly designed the experiment. The task was a driving scenario, where participants were told to do two tasks, a primary task where they had to avoid collison to infrequent object appearing on the side of the road (deer that were about to jump on the road) and a secondary task that was either a button press signalling of puddle colors or slaloming between roadbumps. In the third part both contributors tested the scenario for bugs, and we adjusted the parameters using pilot studies. In the fourth part, Agoston Torok made a second visit to the CRVM lab and they recorded experimental data with 23 participants. Now the user and host labs are jointly working on a future publication that is going to present the results of the experiments.

Conclusions :
Data were recorded from 23 participants, obejctive successfully completed




Project Images :