Facial Interpretation Project: Deriving Functional Axes

Introduction
Techniques such as Principal Components Analysis (PCA) capture the major types of variation within a set of images, and the proportions of these variations present in each image. If, like faces, they have a consistent configuration, techniques using points of correspondence between images can produce a single, non-redundant representation of shape and textural variations. We refer to the patterns of variation as eigenfaces; some examples are shown above.

While the eigenfaces capture all the variation present, and can be used in a variety of applications [see the note on Statistical Appearance Models], they confuse a variety of functional causes. The eigenfaces will code a mixture of lighting, pose, identity and expression variation. Any practical face-interpretation system should attribute vaiation in appearance to the correct cause, and cope with notably unbalanced data-sets, lacking labels on some of the functional axes.

More importantly, the same variation in appearance can have different causes in different contexts. Between individuals, mouth shape may depend upon identity. In a single individual, it depends upon emotional expression or speech production. Further, interactions between the functional axes occur as the images are two-dimensional projections of three-dimensional objects. Thus, any single sub-space we can generate will be a biased estimate of that type of variation. We need to take account of overlapping variation in different sub-spaces, and use as widely based a set of estimates as possible.

Available Spaces
Data-sets are chosen on availability and functional coverage.

A lighting set, 5 images of a single male.

A pose set, 20 images of 5 males, deviations up to from fronto-parallel.

An expression set, 397 images of 19 different sitters, both male and female, making 7 known expressions.

An identity set, 188 images, males and females.

A test set of 98 images of 11 male and female sitters.

The algorithm
The initial estimates of the sub-spaces are determined by PCA. This gives eigenfaces and eigenvalues, the variance on each eigenface. Each face is then coded across the whole set of sub-spaces, giving a set of weights for that image. Since the different sub-spaces overlap, these weights are not unique. Thus we further require the energy of each face's weights be minimized, using ratio of weight to eigenvalue.

This reduces the weights given on low-variance eigenfaces which code similar variations. If these projections on the sub-spaces then generate new sub-spaces, they will be progressively more orthogonal and functional. This procedure repeats until the recoding has no effect, here for 4 iterations.

Some Results
The effectiveness of the algorithm can be assessed by identity recognition on the test set. The images of the individuals vary on lighting, pose and expression. Using nearest-neighbour matching on the identity sub-space, improved coding reduces false matches, as non-identity variation is un-coded.

Recognition is assessed by measuring the frequency that the n closest images are all of the probe individual, varying n. This gives the area of `consistent identity', and across iterations, shows an increase in recognition rates, and a reduction in variance with n. Thus the space is more ordered. In addition, as can be seen in the Pose eigenfaces in the figure, the sub-spaces show reduced contamination from extraneous variance.

Acknowledgments
Funding for this research comes from the EPSRC.

Nick Costen:

Research Courses Clinical Radiology Industrial liaison Contact Us Join Us Search Home

Top Page