Abstract
This chapter focuses on the principles behind methods currently used for face recognition, which have a wide variety of uses from biometrics, surveillance and forensics. After a brief description of how faces can be detected in images, the authors describe 2D feature extraction methods that operate on all the image pixels in the face detected region: Eigenfaces and Fisherfaces first proposed in the early 1990s. Although Eigenfaces can be made to work reasonably well for faces captured in controlled conditions, such as frontal faces under the same illumination, recognition rates are poor. The authors discuss how greater accuracy can be achieved by extracting features from the boundaries of the faces by using Active Shape Models and, the skin textures, using Active Appearance Models, originally proposed by Cootes and Talyor. The remainder of the chapter on face recognition is dedicated such shape models, their implementation and use and their extension to 3D. The authors show that if multiple cameras are used the 3D geometry of the captured faces can be recovered without the use of range scanning or structured light. 3D face models make recognition systems better at dealing with pose and lighting variation.
TopIntroduction
Face recognition is such an integral part of our lives and performed with such ease that we rarely stop to consider the complexity of what is being done. It is the primary means by which people identify each other and so it is natural to attempt to ‘teach’ computers to do the same. The applications of automated face recognition are numerous: from biometric authentication; surveillance to video database indexing and searching.
Face recognition systems are becoming increasingly popular in biometric authentication as they are non-intrusive and do not really require the users’ cooperation. However, the recognition accuracy is still not high enough for large scale applications and is about 20 times worse than fingerprint based systems. In 2007, the US National Institute of Standards and Technology (NIST) reported on their 2006 Face Recognition Vendor Test – FRVT – results (see [Survey, 2007]) which demonstrated that for the first time an automated face recognition system performed as well as or better than a human for faces taken under varying lighting conditions. They also showed a significant performance improvement across vendors from the FRVT 2002 results. However, the best performing systems still only achieved a false reject rate (FRR) of 0.01 (1 in a 100) measured at a false accept rate of 0.001 (1 in one thousand). This translates to not being able to correctly identify 1% of any given database but falsely identify 0.1%. These best-case results were for controlled illumination. Contrast this with the current best results for fingerprint recognition when the best performing fingerprint systems can give an FRR of about 0.004 or less at an FAR of 0.0001 (that is 0.4% rejects at one in 10,000 false accepts) and this has been benchmarked with extensive quantities of real data acquired by US border control and law enforcement agencies. A recent study live face recognition trial at the Mainz railway station by the German police and Cognitec (www.cognitec-systems.de) failed to recognize ‘wanted’ citizens 60% of the time when observing 23,000 commuters a day.
The main reasons for poor performance of such systems is that faces have a large variability and repeated presentations of the same person’s face can vary because of their pose relative to the camera, the lighting conditions, and expressions. The face can also be obscured by hair, glasses, jewellery, etc., and its appearance modified by make-up. Because many face recognitions systems employ face-models, for example locating facial features, or using a 3D mesh with texture, an interesting output of face recognition technology is being able to model and reconstruct realistic faces from a set of examples. This opens up a further set of applications in the entertainment and games industries, and in reconstructive surgery, i.e. being able to provide realistic faces to games characters or applying actors’ appearances in special effects. Statistical modelling of face appearance for the purposes of recognition, also has led to its use in the study and prediction of face variation caused by gender, ethnicity and aging. This has important application in forensics and crime detection, for example photo and video fits of missing persons (Patterson et al., 2007).
Face recognition systems are examples of the general class of pattern recognition systems, and require similar components to locate and normalize the face; extract a set of features and match these to a gallery of stored examples, figure 1. An essential aspect is that the extracted facial features must appear on all faces and should be robustly detected despite any variation in the presentation: changes in pose, illumination, expression etc. Since faces may not be the only objects in the images presented to the system, all face recognition systems perform face detection which typically places a rectangular bounding box around the face or faces in the images. This can be achieved robustly and in real-time.
Figure 1. The basic flow of a recognition system
Key Terms in this Chapter
Eigenimage and Eigenfaces: Face modelling using images as features of a “face space”. An early and successful form of face modelling for recognition.
Statistical Shape, Active Shape and Active Appearance Models: Statistical models of shape and colour used in computer vision. Particularly applicable and effective for face recognition problems.
Biometrics: The science of identifying someone from something they are rather than an identification card or a username/password. Types of biometrics are fingerprints, iris, faces and DNA.
Face Recognition: Automatic recognition of human faces from photographs or videos using a database of know faces. Uses a computer vision and pattern recognition to perform matching of facial features from images to stored “templates” of know faces. Face recognition is one of a number of biometric identification methods.
Face Modelling: The process or taking 2D and 3D images of faces and building a computer model of the faces. This may be a set of facial features and their geometry; the curves of the mouth, eyes, eyebrows, chin and cheeks; or a fully 3D model which incldues depth and colour information. Face modelling can be achieved from either 2D images (static or dynamic) or using 3D range scanning devices.