INTERFACE: Multimodal analysis / synthesis system for human interaction to virtual and augmented environments

• Research unit: EU RFP
• Project number: 99.0675-2

Key words (English)

MPEG-4 player; web-based applications; interactive multimedia; virtual humans animation; synthesis of body emotional gestures; MPEG-7 descriptor schemes.

Alternative project number (English)

EU project number: IST-1999-10036

Research programs (English)

EU-programme: 5. Frame Research Programme - 1.2.4 Essential technologies and infrastructures

Short description (English)

See abstract

Partners and International Organizations (English)

Coordinator: University of Genova (I)

Abstract (English)

MPEG-4 body animation synthesis tool
LIG-EPFL has developed a software tool that uses the MPEG-4 standard format for definition and animation of virtual human bodies in high level.
The body player is a java applet that can be integrated in a web page.
The implemented MPEG-4 player is designed to play 3d animations of virtual humans defined according to the MPEG-4 FBA object specification. It uses as input, uncoded versions of MPEG-4 Body Definition Parameters + Body Deformation Tables to specify the model geometry and MPEG-4 Body Animation Parameters files to specify the animation.
Contribution to the MPEG-7 standard
LIG-EPFL has contributed to the MPEG-7 standard with a descriptor for body emotions. The developed MPEG-7 compliant descriptor scheme defines each parameter used to describe body animation scenes, making special emphasis on the body emotional gestures.
Parameters used to describe the virtual human model and its animation:
The race: American, Caucasian, Ethiopian, Malayan and Mongolian
The gender: male, female and cartoon
The emotion type: sadness, angry, joy, fear disgust, surprise and neutral
The emotion intensity: very low, low, medium, high and very high
The duration of the emotion in minutes and seconds.
Body emotions database and search engine
The search engine developed at LIG allows the search/query of one or more desired MPEG-4 files, more precisely: a BDP and a BAP file. The body emotions database uses the developed MPEG-7 descriptor.
The Body Emotion Search Engine was developed in MS-Visual C++ v.6.0. The code can be easily integrated into a more sophisticated application, the natural evolution would be to merge it with the synthesis tools.
Body emotional gestures demonstration
LIG has provided a demonstration application that shows the synthesis of emotional gestures in different intensity levels depending on the ethnical group that the virtual human belongs to. Europe has been divided into three main regions (northern, central and southern); and 6 basic 'emotional' BAPs are be interpolated according to the gestures intensity expected from each region.

References in databases (English)

Swiss Database: Euro-DB of the
State Secretariat for Education and Research
Hallwylstrasse 4
CH-3003 Berne, Switzerland
Tel. [phone]
Swiss Project-Number: 99.0675-2