Title : The Last Supper Interactive Project (Phase 6-stage 4- Experiment 2)

Project Lead : fischnaller franz From : F.A.B.R.I.CATORS (None)

Dates : from -- to 2015-01-31 00:50:44

Description :

Motivation and objectives :
We, the undersigned are hereby submitting to the call for participation of the VISIONAIR(within the framework of the European Community) for the research and/or experimentation grants that provided access to high level visualization, interaction facilities, resources (physical and virtual) that we dont have access and/or we do not own. Our proposal submission is for the Last Supper Interactive Project-phase 6, stage 4: Haptic interaction, intuitive navigation in stereoscopic immersive virtual environments (Experiment 2) Major goals and tasks that we will like to implement in the VISIONAIR infrastructure facilities will focus in the incorporation of the sense of touch and feel of objects (the haptic sense) for a more sensorial and immersive experience with emphasis to the haptic interactions with virtual models in the context of real time immersive virtual reality user experiences. The experiment will be applied at the schedule time of three (3) weeks. Plus, naturally, before the person from our group arrived to your facilities for the said experimentation and implementation we will go through the pre-production and preparation work necessary to be accomplished from our side in order to ensure that the experimentation and implementation work in your facilities can be successfully processed and accomplished.

Teams :
F.A.B.R.I.CATORS is a creative team is comprised of interdisciplinary groups of highly creative professionals with technical skills, artistic knowledge, innovative concepts, and the capacity to develop and manage projects with interdisciplinary teams. FSAS tailors to user-specific solutions and services for the design and implementation key on hand of: 1. Simulation | Virtual Content | Virtual Worlds Online 2. Digital Media: Animation | 3D Modelling | Graphic Design 3. Interactive Installations: Mixed Medial | Virtual & Augmented Reality 4. Virtual Windows System Integrator 5. Consulting 6. Organization

Dates :
starting date : 06 October, 2014
ending date : 09 December, 2015

Facilities descriptions :
http://visionair-browser.g-scop.grenoble-inp.fr/visionair/Browser/Catalogs/MEXICO.FR.html

Recordings & Results :
The Last Supper Interactive (LSI) project is a real-time- immersive interactive storytelling and virtual narrative stereo application in high-definition (3D-4K) based on the Last Supper (Italian: L'Ultima Cena); a late 15th century mural painting by Leonardo da Vinci located in the refectory of the Convent of Santa Maria delle Grazie, in Milan, Italy [1]. LSI application can run in an ultra-high resolution visualization system - 4k stereo display. Initially LSI was conceived as a nine-phase development.

Conclusions :
The refectory and a 3D scene representing the painting have been created using Maya. Unity has been used to perform the visualization and to manage the virtual interactions. A python program (cveHaptic) has been developed to manage the Haptic arm. This program allow us to know the position of the arm as a 4x4 transform matrix, at the chosen frequency, and to give 6D force feedback orders back to the arm. The program use the Haption Virtuose API supplied with the arm.Another python program (cveUnity) has been developed to make the communication between cveHaptic and the Unity scene. This software use network sockets for the communication. It create a local socket server in python, then launch the Unity executable representing the virtual interactive scene, and start a network ccommunication between the executable and the local server. So both side (server and unity client) can send messages to the other side at any time, as there is a message listener running both side. On the Unity scene, a network client listener had to be implemented too, so that the unity executable could connect to our python server. It had been developed as a c# unity script. Both python server and c# client share the same message protocol that we created. Finally, synchronization between our two python program is handle by our high level CVE software, wish allow to run several plugins, possibly in several machines, and to share a model between the different plugins. In fact, cveHaptic and cveUnity are both CVE plugins, and that's how the experiment has been implemented. Interactions: The user can move in the scene like he was walking, by rolling the wheel. An avatar represent his hand in the scene, and follow the user real hand position. Force feedback is returned to the user if his hand hit an obstacle, so that he cannot go forward. Scenario: The user start in the middle of the chapel, on the ground, seeing the painting. When he go forward, an invisible ramp allow him to arrive face to face with Jesus scene representing the painting room, but empty, except for light at the apostles positions. When the user collide an invisible furniture or an invisible apostle, it appears with a sound and an haptic feedback, so that the user can recreate the entire scene by touching each element. Input: Haption haptic arm 6 dof:the arm position give us the 6d position of the user arm at any time (frequency used : 100 Hz); a 2D wheel on the handle is use to move in the room; a button is use to select objects Output: 6K power wall for a ultra high definition visualization, sound feedbacks, haptic feedbacks of the Haption arm. The expected outcome of this experiment was successfully delivered. The most important direction for future work would be to further incorporate haptic technology , i.e. a sense of touch: haptic perception and interaction. With and within the virtual objects and environments e.g. Albertis theorem virtual tool,Christ Cup, Christ and the Apostles, etc




Project Images :