Handbook of Face Recognition Stan Z. Li (cid:2) Anil K. Jain Editors Handbook of Face Recognition Second Edition Editors StanZ.Li AnilK.Jain InstituteofAutomation,CenterBiometrics Dept.ComputerScience&Engineering Research&Security MichiganStateUniversity ChineseAcademyofScience EastLansing,MI48824-1226 Room1227,No.95ZhongguancunEastRd USA Beijing100190 [email protected] People’sRepublicofChina [email protected] ISBN978-0-85729-931-4 e-ISBN978-0-85729-932-1 DOI10.1007/978-0-85729-932-1 SpringerLondonDordrechtHeidelbergNewYork BritishLibraryCataloguinginPublicationData AcataloguerecordforthisbookisavailablefromtheBritishLibrary LibraryofCongressControlNumber:2011936022 ©Springer-VerlagLondonLimited2011 Apartfromanyfairdealingforthepurposesofresearchorprivatestudy,orcriticismorreview,asper- mittedundertheCopyright,DesignsandPatentsAct1988,thispublicationmayonlybereproduced, storedortransmitted,inanyformorbyanymeans,withthepriorpermissioninwritingofthepublish- ers,orinthecaseofreprographicreproductioninaccordancewiththetermsoflicensesissuedbythe CopyrightLicensingAgency.Enquiriesconcerningreproductionoutsidethosetermsshouldbesentto thepublishers. Theuseofregisterednames,trademarks,etc.,inthispublicationdoesnotimply,evenintheabsenceofa specificstatement,thatsuchnamesareexemptfromtherelevantlawsandregulationsandthereforefree forgeneraluse. Thepublishermakesnorepresentation,expressorimplied,withregardtotheaccuracyoftheinformation containedinthisbookandcannotacceptanylegalresponsibilityorliabilityforanyerrorsoromissions thatmaybemade. Coverdesign:VTeXUAB,Lithuania Printedonacid-freepaper SpringerispartofSpringerScience+BusinessMedia(www.springer.com) Preface Facerecognitionisoneofthemostimportantabilitiesthatweuseinourdailylives. There are several reasons for the growing interest in automated face recognition, including rising concerns for public security, the need for identity verification for physicalandlogicalaccess,andtheneedforfaceanalysisandmodelingtechniques in multimedia data management and digital entertainment. Research in automatic face recognition started in the 1960s. Recent years have seen significant progress inthisareaandanumberoffacerecognitionandmodelingsystemshavebeende- velopedanddeployed.However,accurateandrobustfacerecognitionstilloffersa numberofchallengestocomputervisionandpatternrecognitionresearchers,espe- ciallyunderunconstrainedenvironments. Thisbookiswrittenwithtwoprimarymotivations.Thefirstistocompilemajor approaches,algorithms,andtechnologiesavailableforautomatedfacerecognition. Thesecondistoprovideareferencebooktostudents,researchers,andpractitioners. Thebookisintendedforanyonewhoplanstoworkinfacerecognitionorwho wants to become familiar with the state-of-the-art in face recognition. It also pro- videsreferencesforscientistsandengineersworkinginimageprocessing,computer vision, biometrics and security, computer graphics, animation, and the computer gameindustry.Thematerialcontainedinthebookfitsthefollowingcategories:ad- vancedtutorial,state-of-the-artsurvey,andaguidetocurrenttechnology. This second edition consists of twenty seven chapters, with additions and up- dates from the sixteen chapters in the first edition. It covers all the subareas and majorcomponentsnecessaryfordevelopingfacerecognitionalgorithms,designing operationalsystems,andaddressingrelatedissuesinlargescaleapplications.Each chapterfocusesonaspecifictopicorsystemcomponent,introducesbackgroundin- formation,reviewsup-to-datetechniques,presentsresults,andpointsoutchallenges andfuturedirections. Thetwentysevenchaptersaredividedintofourpartsaccordingtothemainprob- lems addressed. Part I, Face Image Modeling and Representation, consists of ten chapters,presentingtheoriesinfaceimagemodelingandfacialfeaturerepresenta- tion.PartII,FaceRecognitionTechniques,alsoconsistsoftenchapters,presenting techniquesforfacedetection,landmarkdetection,andfacerecognitioninstaticface v vi Preface images,invideo,innon-visiblespectrumimages,andin3D.PartIII,Performance Evaluation:MachinesandHumans,consistsofthreechapters,presentingmethods andprogramsforfacerecognitionevaluationandalsostudiesandcomparisonswith humanperformance.PartIV,FaceRecognitionApplications,consistsoffourchap- ters,presentingvariousapplicationsoffacerecognitionandrelatedissues. A project like this requires the efforts and support of many individuals and or- ganizations.Firstofall,wewouldliketothankalltheauthorsfortheiroutstanding contributionswhichmadethiseditionpossible.WealsothankWayneWheelerand Simon Rees, the Springer editors for their support and patience during the course ofthisproject.Thanksarealsoduetoanumberofindividualswhohaveassistedus duringtheeditingphaseofthisproject,includingShikunFeng,ShengcaiLiao,Xi- angsheng Huang, Brendan Klare, Unsang Park, Abhishek Nagar, and not the least Kim Thompson for her careful proofreading of the manuscript. Stan Z. Li would like to acknowledge the support of the Chinese National Natural Science Foun- dationProject#61070146,theNationalScienceandTechnologySupportProgram Project#2009BAK43B26,theAuthenMetricR&DFunds,andtheTABULARASA project (http://www.tabularasa-euproject.org) under the Seventh Framework Pro- grammeforresearchandtechnologicaldevelopment(FP7)oftheEuropeanUnion (EU), grant agreement #257289. Anil Jain’s research was partially supported by theWCU(WorldClassUniversity)programfundedbytheMinistryofEducation, ScienceandTechnologythroughtheNationalResearchFoundationofKorea(R31- 10008)totheBrain&CognitiveEngineeringDepartment,KoreaUniversitywhere heisanAdjunctProfessor. Beijing,People’sRepublicofChina StanZ.Li EastLansing,USA AnilK.Jain Contents 1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1 StanZ.LiandAnilK.Jain PartI FaceImageModelingandRepresentation 2 FaceRecognitioninSubspaces . . . . . . . . . . . . . . . . . . . . . 19 GregoryShakhnarovichandBabackMoghaddam 3 FaceSubspaceLearning . . . . . . . . . . . . . . . . . . . . . . . . . 51 WeiBianandDachengTao 4 LocalRepresentationofFacialFeatures . . . . . . . . . . . . . . . . 79 Joni-KristianKämäräinen,AbdenourHadid,andMattiPietikäinen 5 FaceAlignmentModels . . . . . . . . . . . . . . . . . . . . . . . . . 109 PhilTresadern,TimCootes,ChrisTaylor,andVladimirPetrovic´ 6 MorphableModelsofFaces . . . . . . . . . . . . . . . . . . . . . . . 137 ReinhardKnothe,BrianAmberg,SamiRomdhani,VolkerBlanz,and ThomasVetter 7 IlluminationModelingforFaceRecognition . . . . . . . . . . . . . . 169 RonenBasriandDavidJacobs 8 FaceRecognitionAcrossPoseandIllumination . . . . . . . . . . . . 197 RalphGross,SimonBaker,IainMatthews,andTakeoKanade 9 SkinColorinFaceAnalysis . . . . . . . . . . . . . . . . . . . . . . . 223 J.BirgittaMartinkauppi,AbdenourHadid,andMattiPietikäinen 10 FaceAgingModeling . . . . . . . . . . . . . . . . . . . . . . . . . . . 251 UnsangParkandAnilK.Jain vii viii Contents PartII FaceRecognitionTechniques 11 FaceDetection . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 277 StanZ.LiandJianxinWu 12 FacialLandmarkLocalization. . . . . . . . . . . . . . . . . . . . . . 305 XiaoqingDingandLitingWang 13 FaceTrackingandRecognitioninVideo . . . . . . . . . . . . . . . . 323 RamaChellappa,MingDu,PavanTuraga,andShaohuaKevinZhou 14 FaceRecognitionataDistance . . . . . . . . . . . . . . . . . . . . . 353 FrederickW.Wheeler,XiaomingLiu,andPeterH.Tu 15 FaceRecognitionUsingNearInfraredImages . . . . . . . . . . . . . 383 StanZ.LiandDongYi 16 MultispectralFaceImagingandAnalysis . . . . . . . . . . . . . . . 401 AndreasKoschan,YiYao,HongChang,andMongiAbidi 17 FaceRecognitionUsing3DImages . . . . . . . . . . . . . . . . . . . 429 I.A.Kakadiaris,G.Passalis,G.Toderici,E.Efraty,P.Perakis,D.Chu, S.Shah,andT.Theoharis 18 FacialActionTracking . . . . . . . . . . . . . . . . . . . . . . . . . . 461 JörgenAhlbergandIgorS.Pandzic 19 FacialExpressionRecognition . . . . . . . . . . . . . . . . . . . . . . 487 YingliTian,TakeoKanade,andJeffreyF.Cohn 20 FaceSynthesis. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 521 YangWang,ZichengLiu,andBainingGuo PartIII PerformanceEvaluation:MachinesandHumans 21 EvaluationMethodsinFaceRecognition . . . . . . . . . . . . . . . . 551 P.JonathonPhillips,PatrickGrother,andRossMicheals 22 DynamicAspectsofFaceProcessinginHumans . . . . . . . . . . . . 575 HeinrichH.Bülthoff,DouglasW.Cunningham,andChristianWallraven 23 FaceRecognitionbyHumansandMachines . . . . . . . . . . . . . . 597 AliceJ.O’Toole PartIV FaceRecognitionApplications 24 FaceRecognitionApplications. . . . . . . . . . . . . . . . . . . . . . 617 ThomasHuang,ZiyouXiong,andZhenqiuZhang 25 LargeScaleDatabaseSearch . . . . . . . . . . . . . . . . . . . . . . 639 MichaelBrauckmannandChristophBusch Contents ix 26 FaceRecognitioninForensicScience . . . . . . . . . . . . . . . . . . 655 NicoleA.Spaun 27 PrivacyProtectionandFaceRecognition . . . . . . . . . . . . . . . . 671 AndrewW.SeniorandSharathchandraPankanti Index . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 693