ABOUT FACE: COMPUTERGRAPHIC SYNTHESIS AND MANIPULATION OF FACIAL IMAGERY by Peggy Weil B.A., Radcliffe College, Harvard University 1976 Submitted in Partial Fulfillment of the requirements for the Degree of Master of Science in Visual Studies at the Massachusetts Institute of Techonolgy September 1982 Massachusetts Institute of Technology 1982 Signature of Author........ ... . . .-.-----.. ---------------.. Department of Architecture August 6, 1982 Certified by. ... ............. Andr Lippman, Assistant Profe ssor f Media Technology Thesis Supervisor Accepted by............./---- Nicholas P. Negroponte Chairman, Departmental Committee on Graduate Students Rot01 MA SSACH63"ETTS INSiiT7 OF TECHNOLOGY SEP 21 1982 UIBRARIES Room 14-0551 77 Massachusetts Avenue M Libraries Cambridge, MA 02139 Ph: 617.253.2800 Email: [email protected] Document Services http://Iibraries.mit.edu/docs DISCLAIMER NOTICE The accompanying media item for this thesis is available in the MIT Libraries or Institute Archives. Thank you. ABOUT FACE: COMPUTERGRAPHIC SYNTHESIS AND MANIPULATION OF FACIAL IMAGERY by Peggy Weil Submitted to the Department of Architecture on August 6,1982 in partial fulfillment of the requirements for the degree of Master of Science in Visual Studies ABSTRACT A technique of pictorially synthesizing facial imagery using optical videodiscs under computer control is described. Search, selection and averagin processes are performed on a catalogue of whole faces and facial features to yield a composite, expressive, recognizable face. An immediate application of this techique is the reconstruction of a particular face from memory for police identification, thus the project is called, IDENTIDISC. Part I-FACEMAKER describes the production and implementation of the IDENTIDISC system to produce composite faces. Part II-EXPRESSIONMAKER describes animation techniques to add expression and motion to composite faces. Expression sequences are manipulated to make 'anyface' make any face. Historical precedents of making facial composites, theories of facial recognition, classification and expression are also discussed. This thesis is accompanied by two copies of FACEMAKER-III, an optical videodisc produced at the Architecture Machine Group in 1982. The disc can be played on an optical videodisc player. The length is approximately 15,0000 frames. Frame numbers are indicated in the text by [ 1. Thesis Supervisor: Andrew Lippman Title: Assistant Professor of Media Technology The work reported.herein was supported by the Cybernetics Technology Division of the Defense Advanced Research Projects Agency, under Contract No. MDA-9.03-8-C-0097. 2. CONTENTS I. FACEMAKER 1. Introduction and Objectives 6 2. Facial Recognition and Memory 9 3. Facial Composites 14 4. Classification of Faces 18 5. IDENTIDISC 21 6. Facemaking System 34 7. Facemaking Procedures 36 8. Examples of Composite Faces 40 II. EXPRESSIONMAKER 9. Objective 47 10. Facial Expression 49 10.1 Theories of Facial Expression 50 10.2 Classification 53 10.21 Anatomical Notation 56 10.22 Psychological Demonstrational Models 62 10.23 Recording by Ethnologists 64 11. Portrayal of Facial Expression 66 12. Expressionmaker Afterword 81 Appendix I: FACEMIAKER-III Disclog 84 Appendix II: Facem-aking Procedures 86 References 87 Acknowledgements 91 3. I. FACEMAKER 4. FACEMAKER is dedicated to all the faces that added another point of view. 5. 1.0 Introduction and Objectives Our general impressions are founded upon blended memories. - Francis Galton, Inquiries into Human Faculty The purpose of recall implies at least the possibility of causing two impressions to appear as quasi-Zikeness... Without imagination there would be no resemblance between things. - Michel FoucouZt, The Order of Things FACEMAKER is a technique for pictorially synthesizing facial images by intensity averaging a selection of photographs of registered faces which are stored on optical videodisc. The stated, and most immediate application of this technique is the reconstruction of a particular face from memory, as in the case of police identification of a suspect from the description of an assailant. The objective of this research is to demonstrate a computerized Identification system to generate a target face by averaging photographs to form a composite. The procedure is to average (either photographically or compu- tationally) together faces which are judged to be similar to the "remembered" face (hereafter referred to as the "target"). Photo- graphic averaging is done by making multiple exposures in the camera, computational averaging is done by averaging intensity values of pixels in successive digitized images. In the averaging process, the features of the component faces which bear a resemblance to the target 6. tend to become emphasized, as the features which are dissimilar tend to be lost-effectively a pictorial noise cancellation procedure. The result is an approximate likeness to the target; a pictorial list of traits for which the components were chosen. The Identidisc system replaces photographic methods of averaging with electronic techniques to allow for dynamic interaction, breadth and to accommodate classifi- cation procedures. The challenge that FACEMAKER poses to feature-based identifi- cation systems is based on the assertion that it is compatable with the way we recognize and remember faces. Chapter Two examines research regarding facial recognition and recall. Chapter Three reviews the historical precedents of making photographic facial composites-a practice that dates back to the early days of photography. Chapter Four considers theories of facial classification-relevant to the task of organizing several thousand photographs of faces. Chapters Five, Six, Seven and Eight describe the IdentiDisc System; they elaborate and illustrate the procedure for producing composite faces. EXPRESSIONMAKER forms the second part of this thesis. The objective of EXPRESSIONMAKER is to add expression and motion to compo- site faces by integrating techniques described above with animation. The text is organized similarly to FACEMAKER; and could almost be con- sidered a separate body of work. Chapter Nine is an introduction. Chapter Ten recounts theoretical approaches to the study of facial expression-conjectures as to its cause and origin and attempts at classification. Chapter Ten discusses the portrayal of facial expres- sion in art, theater and animation. Chapter Eleven is an account -7 I~. of my efforts to animate faces within the mixed contexts of ongoing research in teleconferencing and facial identification. 2.0 Facial Recognition and Memory The issues of facial recognition and recall are clearly pertinent to the creation of a facial identification system. There is a broad spectrum of research concerned with memory for faces; ranging from psychological literature investigating the underlying perceptual mechanisms for pictorial stimuli to engineering questions regarding the task of machine recognition of faces from mathematical descriptions. Of interest to this project are studies of the role of photographs to identification procedures, the effect of the length of exposure for recall, and discussions of factors which have been shown to affect recognition. Following is a brief survey of some of the literature with references to these issues. The ability to reconstruct a face may not be commensurate with the ability to recognize one so these studies must be interpreted as suggestive rather than conclusive within the given context. The difference between experimental laboratory pro- cedures and actual situations where an identification system might be used highlight areas which remain to be studied; specifically, Ellis et.al. (1975) suggests that research be done to determine the affect of emotional excitement (i.e., the trauma of a criminal encounter) on recognition, and on the discrepancies between recognition of photo- graphs and real people. 9.