MESH: Modeling expressions and shapes of human heads

• Research unit: EU RFP
• Project number: 99.0230-1

Key words (English)

Face animation; texture mapping; texture synthesis; 3D modeling

Alternative project number (English)

EU project number: IST-1999-10443

Research programs (English)

EU-programme: 5. Frame Research Programme - 1.2.3 Multimedia content and tools

Short description (English)

See abstract

Partners and International Organizations (English)

Coordinator: Duran (F)

Abstract (English)

~JJo;; J~J~' ~V(1J Vi Din uunng IllIS penoa nasl>een the final refinements to the algorithms to be put into the fIrSt prototype tools and the creation of these initial tools as a Maya plug-in. This work had to be carried out both for the low-quality version of the hair synthesis tool and the viseme animation tool.
Hair modelling
During the second year, ETH has experimented further with multiple layers of synthesized hair texture, in order to give an illusion of volumetric hair synthesis at much lower cost of rendering. Experiments were carried out with different levels of brightness and transparency for the layers. A second focus was on mapping hair texture onto the Mesh standard head topology. Given the local hair orientations specified by a hair style, the hair textures of the layers will follow this orientation patterns. ETH has created a simple hair style for early experimentations and have later switched to orientation fields supplied by Duran. Last but not least, the first version of the Maya plug-in for low- quality hair modeling was developed.
Modeling of face dynamics
During the second year, ETH has finalized the analysis of 3D viseme shapes for a first batch of faces (processing the data takes a lot of time, even with the help of the tracker that ETH and Eyetronics have developed earlier during the project, and the MeshModeler tool that Eyetronics has made available to map the raw 3D data to the Mesh topology). The expressions that could be extracted from these data were analyzed, both in terms of Principal Components and Independent Components. The Independent Components were chosen as the final basis, as they came out to be more intuitive to control. The different visemes that ETH had identified earlier on in the project were expressed in terms of this basis. ETH have also further elaborated their co-articulation strategy. Whereas vocals and labial consonants are visited precisely, the system is only attracted to different degrees to the visemes of other consonants. A first version of the Maya plug-in for speech animation was implemented. In contrast to the earlier stages of development, it is now possible to efficiently animate any face that is delivered in the prescribed Mesh format.
BBW06.200I 2
-''--
ETH. has carrie~ out an experiment where the viseme plug-in was used for speech-based animation The mput here IS speech, either as an audio file, or as text from which text is generated with the text~ to-speech program SVOX, an ETH spin-off with whom the ETH group involved in Mesh had alread collaborated ~fore ~o get the nece~sary p~on~m~-to-~isem~ transcription done efficiently: Phonem~
.are translated mto ~Isemes plus theIr relative tilDlng (mcludmg silence). The results of this initial test c~e out to be satisfactory, but no professional animators have been asked for their opinion at this pomt.

References in databases (English)

Swiss Database: Euro-DB of the
State Secretariat for Education and Research
Hallwylstrasse 4
CH-3003 Berne, Switzerland
Tel. [phone]
Swiss Project-Number: 99.0230-1