High level visual perception of faces and non-rigid objects
[This research came to a conclusion in Apr. 2002]

Shigeru Akamatsu^*#, Keisuke Kinoshita^*, Miyuki Kamachi^*

Human visual system can recognize shape and texture of a 3D object which is moving as well as deforming in a complex environment, and even what is meant by such dynamic characteristics of the 3D object: its motion and deformation. The goal of our research group is to understand and model such a high level process in visual perception of non-rigid 3D objects. As an example, we mainly focus on the perception of human facial expressions. Face dynamically deforms its shape depending on the expressions and then induces changes in visual information that we get through our eyes. We human process such information and recognize the expressions: anger, sad, happy, etc. This is a mapping from the physical characteristics of the object to the psychological descriptions of expressions. Building a computational model of such mapping is worth challenging and may contribute to neuro-physiological studies of high level vision.

To solve this problem, two related studies have to be combined: computer vision and psychology. Computer vision techniques are used to extract 3D physical features in the image sequence. Optical flows and geometrical relationships are good cues to know dynamically deforming shape of the faces. On the other hand, psychological experiments are conducted to collect data of how humans perceive facial expressions from faces of different race and identity under various viewing conditions such as orientations. We emphasize to make a "bridge" between these two: the physical information and the perceptual information.