Title : Constructive Conversations - Modelling Interaction Space, Gesturing, and Gazing in natural interactive encounters with humans and robots

Project Lead : Kristiina Jokinen From : University of Helsinki (None)

Dates : from -- to 2014-02-18 16:32:32

Description :

Motivation and objectives :
Natural language is used to exchange information, and the effective transfer of information is often taken as the main criterion for the success of interaction. Especially in the context of automatic services, the delivery of reliable and relevant information is an important goal for the design of such systems. Recently, however, one of the challenges for designing interactive systems has been identified as being related to social aspects of interactions: how to engage the partner in the interaction and keep their interest up so that the speaker can either deliver the message they intend to deliver, or can provide rapport and affection so as to create a mutual bond and an understanding relationship. Our interest in studying conversational engagement goes back to intelligent systems and interaction technology, where engagement is used to describe the users willingness and involvement in the interaction with the automatic interactive system. If it is possible to measure the interlocutors engagement level, it is easier to adjust the systems conversational strategies accordingly. One of the ways to measure engagement is to study the interlocutors' paralinguistic singnalling concerning their interest, level of understanding, and focus of attention. For instance, with motion tracker technology such issues as the participants' distance from each other can be measured, and combined with the research questions of the participants' engagement in the interaction, the novel technology can provied important and useful objective information of the interaction space and the participants' control over the space around them. This research will not enhance only our understanding of the comfortable communication between humans or between humans and virtual agents, but will also allow us to build models for the automatic recognition and management of issues related to appropriate and smooth communication.

Teams :
3I (Intelligent Interactice Informatics) Group is a research group in the Institute of Behavioural Sciences at the University of Helsinki. Our research focuses on intelligent interaction and information systems, and our areas of expertise include multimodal interaction, interaction management, paralinguistic communication, corpus collection and annotation, machine learning techniques, clustering and classification of linguistic data, learning and interaction. We currently work on multimodal corpus analysis (top-down human annotations and observations as well as bottom-up signal analysis) on naturally flowing human-human conversations and first encounter interactions, especially focussing on eye-gaze, face, and hand gestures, and their use in signalling turn-taking and feedback, in order to develop models for interaction techniques and strategies.

Dates :
starting date : 28 January, 2013
ending date : 09 February, 2013

Facilities descriptions :
http://visionair-browser.g-scop.grenoble-inp.fr/visionair/Browser/Catalogs/3DICC.HU.html

Recordings & Results :
The TNA project was successful from several points of view: while discussing future research goals, we were able to submit a joint proposal to an EU research grant and we have also explored possibilities to integrate the Nao WikiTalk application into the Virtual Collaboration Arena.

Conclusions :
We successfully submitted the EU FET-Open Xtrack proposal in time. We had fruitful discussions on the Nao WikiTalk application and its possible enhancements for future presentations at the CogInfoCom conference. We developed a rudimentary implementation in which the VirCA system was able to access wikipedia articles for users. Future enhancements of this application point towards the full integration of Nao WikiTalk into VirCA.




Project Images :





Other project resources :

Report of the visit-20pages_new-final.pdf