Lecture Notes in Computer Science 3979 CommencedPublicationin1973 FoundingandFormerSeriesEditors: GerhardGoos,JurisHartmanis,andJanvanLeeuwen EditorialBoard DavidHutchison LancasterUniversity,UK TakeoKanade CarnegieMellonUniversity,Pittsburgh,PA,USA JosefKittler UniversityofSurrey,Guildford,UK JonM.Kleinberg CornellUniversity,Ithaca,NY,USA FriedemannMattern ETHZurich,Switzerland JohnC.Mitchell StanfordUniversity,CA,USA MoniNaor WeizmannInstituteofScience,Rehovot,Israel OscarNierstrasz UniversityofBern,Switzerland C.PanduRangan IndianInstituteofTechnology,Madras,India BernhardSteffen UniversityofDortmund,Germany MadhuSudan MassachusettsInstituteofTechnology,MA,USA DemetriTerzopoulos UniversityofCalifornia,LosAngeles,CA,USA DougTygar UniversityofCalifornia,Berkeley,CA,USA MosheY.Vardi RiceUniversity,Houston,TX,USA GerhardWeikum Max-PlanckInstituteofComputerScience,Saarbruecken,Germany Thomas S. Huang Nicu Sebe Michael S. Lew Vladimir Pavlovic´ Mathias Kölsch Aphrodite Galata Branislav Kisacˇanin (Eds.) Computer Vision in Human-Computer Interaction ECCV 2006 Workshop on HCI Graz, Austria, May 13, 2006 Proceedings 1 3 VolumeEditors ThomasS.Huang E-mail:[email protected] NicuSebe E-mail:[email protected] MichaelS.Lew E-mail:[email protected] VladimirPavlovic´ E-mail:[email protected] MathiasKölsch E-mail:[email protected] AphroditeGalata E-mail:[email protected] BranislavKisacˇanin E-mail:[email protected] LibraryofCongressControlNumber:2006925105 CRSubjectClassification(1998):I.4,I.5,I.3,H.5.2,K.4.2 LNCSSublibrary:SL6–ImageProcessing,ComputerVision,PatternRecognition, andGraphics ISSN 0302-9743 ISBN-10 3-540-34202-8SpringerBerlinHeidelbergNewYork ISBN-13 978-3-540-34202-1SpringerBerlinHeidelbergNewYork Thisworkissubjecttocopyright.Allrightsarereserved,whetherthewholeorpartofthematerialis concerned,specificallytherightsoftranslation,reprinting,re-useofillustrations,recitation,broadcasting, reproductiononmicrofilmsorinanyotherway,andstorageindatabanks.Duplicationofthispublication orpartsthereofispermittedonlyundertheprovisionsoftheGermanCopyrightLawofSeptember9,1965, initscurrentversion,andpermissionforusemustalwaysbeobtainedfromSpringer.Violationsareliable toprosecutionundertheGermanCopyrightLaw. SpringerisapartofSpringerScience+BusinessMedia springer.com ©Springer-VerlagBerlinHeidelberg2006 PrintedinGermany Typesetting:Camera-readybyauthor,dataconversionbyScientificPublishingServices,Chennai,India Printedonacid-freepaper SPIN:11754336 06/3142 543210 Preface The interests and goals of HCI (human–computer interaction) include under- standing, designing, building, and evaluating complex interactive systems in- volving many people and technologies. Developments in software and hardware technologies are continuously driving applications in supporting our collabora- tive and communicative needs as social beings, both at work and at play. At the sametime,similardevelopmentsarepushing the human–computerinterface beyond the desktop and into our pockets, streets, and buildings. Developments in mobile, wearable,and pervasivecommunications and computing technologies provide exciting challenges and opportunities for HCI. The present volume represents the proceedings of the HCI 2006 Workshop that was held in conjunction with ECCV 2006 (European Conference on Com- puter Vision) in Graz,Austria.The goalof this workshopwas to bring together researchers from the field of computer vision whose work is related to human– computer interaction. We solicited original contributions that address a wide range of theoretical and application issues in human–computer interaction. Wewereverypleasedbytheresponseandhadadifficulttaskofselectingonly 11 papers (out of 27 submitted) to be presented at the workshop.The accepted papers were presented in four sessions, as follows: Face Analysis – In their paper “Robust Face Alignment Based On Hierarchical Classifier Network”authorsLiZhang,HaizhouAi,andShihongLaobuildahierarchi- cal classifier network that connects face detection and face alignment into a smooth coarse-to-fine procedure. Thus a robust face alignment algorithm onfaceimageswithexpressionandposechangesisintroduced.Experiments are reported to show its accuracy and robustness. – In“EigenExpressApproachinRecognitionofFacialExpressionusingGPU” authorsQiWu,MingliSong,JiajunBu,andChunChenpresentanefficientfa- cialexpressionrecognitionsystembasedonaGPU-basedfilterforpreprocess- ingandEigenExpressandModifiedHausdorffdistanceforclassification. – In “Face Representation Method Using Pixel-to-Vertex Map for 3D Model- Based Face Recognition” authors Taehwa Hong, Hagbae Kim, Hyeonjoon Moon,YonggukKim,JongweonLee,andSeungbinMoondescribea3Dface representation algorithm to reduce the number of vertices and optimize its computationtime.Theyevaluatetheperformanceoftheproposedalgorithm withtheKoreanfacedatabasecollectedusingastereo-camera-based3Dface capturing device. – In “Robust Head Tracking with Particles Based on Multiple Cues Fusion” authorsYuanLi,HaizhouAi,ChangHuang,andShihongLaopresentafully automaticandhighlyrobustheadtrackingalgorithmthatfusesthefacecues fromareal-timemultiviewfacedetectionwithcolorspatiogramandcontour VI Preface gradient cues under a particle filter framework.Experiments show that this algorithm is highly robust against target position, size, and pose change, as well as unfavorable conditions such as occlusion, poor illumination, and cluttered background. Gesture and Emotion Recognition – In “Vision-Based Interpretation of Hand Gestures for Remote Control of a Computer Mouse” authors Antonis A. Argyros and Manolis I. A. Lourakis presentahuman–computerinteractionsystemthatiscapableofrecognizing handgesturesandofinterpretingthemtoremotelycontrolacomputermouse. This workis basedontheir previous workon2D and3Dtrackingofcolored objects.Twodifferentgesturalvocabulariesareinvestigated,basedon2Dand 3D hand information, respectively. Experiments are used to compare these vocabulariesintermsofefficiency,robustness,reliability,andeaseofuse. – In “Computing Emotion Awareness ThroughFacial Electromyography”au- thors Egon van den Broek, Marleen Schut, Joyce Westerink, Jan van Herk, and Kees Tuinenbreijer use coarse time windows to discriminate between positive, negative, neutral, and mixed emotions. They use six parameters (i.e., mean, absolute deviation, standard deviation, variance, skewness, and kurtosis) of three facial EMGs: zygomaticus major, corrugator supercilii, and frontalis. The zygomaticus major is shown to discriminate excellently between the four emotion categories and, consequently, can facilitate em- pathic HCI. Event Detection – In “Silhouette-Based Method for Object Classification and Human Action Recognition in Video” authors Yig˘ithan Dedeog˘lu, B. Ug˘ur To¨reyin, Ug˘ur Gu¨du¨kbay, and A. Enis C¸etin present an instance-based machine learning algorithmand a system for real-time object classificationandhuman action recognition which makes use of object silhouettes. An adaptive background subtraction model is used for object segmentation. A supervised learning methodbasedontemplatematchingisadoptedtoclassifyobjectsintoclasses like human, human group, and vehicle, and human actions into predefined classes like walking, boxing, and kicking. – In“VoiceActivityDetectionUsingWavelet-BasedMultiresolutionSpectrum and Support Vector Machines and Audio Mixing Algorithm” authors Wei Xue, Sidan Du, Chengzhi Fang, and Yingxian Ye present a voice activity detection (VAD) algorithmand efficient speech mixing algorithmfor a mul- timedia conference.The proposedVAD uses MFCCofmultiresolutionspec- trum as features and classifies voice by support vector machines (SVM). – In “Action Recognition in Broadcast Tennis Video Using Optical Flow and SupportVectorMachine”authorsGuangyuZhu,ChangshengXu,WenGao, andQingmingHuangpresenta novelapproachto recognizethe basicplayer actions in broadcast tennis video where the player is only about 30 pixels Preface VII tall. A new motion descriptor based on optical flow is proposed where the optical flow is treated as spatial patterns of noisy measurements instead of precisepixeldisplacements.Supportvectormachineisemployedtotrainthe action classifier. Applications – In “FaceMouse — A Human-Computer Interface for Tetraplegic People” authors Emanuele Perini, Simone Soria, Andrea Prati, and Rita Cucchiara propose a new human–machine interface particularly conceived for people with severe disabilities (specifically tetraplegic people), that allows them to interact with the computer. They have studied a new paradigm called “derivativeparadigm,”wheretheusersindicatethedirectionalongwhichthe mouse pointer must be moved. The system that uses this paradigmconsists of a common, low-cost webcam and a set of computer vision techniques developedtoidentifythepartsoftheuser’sfaceandexploitthemformoving the pointer. – In “Object Retrieval by Query with Sensibility Based on the KANSEI- Vocabulary Scale” authors Sunkyoung Baek, Myunggwon Hwang, Miyoung Cho, Chang Choi, and Pankoo Kim propose the KANSEI-Vocabulary Scale by associating human sensibilities with shapes among visual information. They construct the object retrieval system for evaluation of their approach and are able to retrieve object images with the most appropriate shape in terms of the query’s sensibility. We would like to thank the contributing authors and Springer’s LNCS team fortheir help inpreparationofthe workshopproceedings.Therewouldnotbe a workshoptobeginwithhadit notbeenforthe invaluablehelpwe receivedfrom the Program Committee members (listed later in the book) and their careful reviews of submitted papers. The review process has been facilitated by the Conference Management Toolkit, a free service provided by Microsoft Research (http://msrcmt.research.microsoft.com/cmt).We would also like to thank theChairsoftheECCV2006ConferenceinGraz,Austria,fortheirsupportand help.Finally,wewouldliketothankourcorporatesponsor,DelphiCorporation, for generous support of the workshop. May 2006 T.S. Huang Graz, Austria N. Sebe M.S. Lew V. Pavlovi´c M. Ko¨lsch A. Galata B. Kisaˇcanin HCI 2006 Chairs Organization HCI2006(WorkshoponHuman–ComputerInteraction)washeldinconjunction with ECCV2006(EuropeanConferenceonComputer Vision), on13May 2006, in Graz, Austria. Organizing Committee General Chair Thomas S.Huang (University of Illinois at Urbana-Champaign, USA) ProgramChairs NicuSebe(UniversityofAmsterdam,Netherlands) MichaelS.Lew(UniversityofLeiden,Netherlands) Vladimir Pavlovi´c(Rutgers University, USA) MathiasKo¨lsch(NavalPostgraduateSchool,USA) Publicity Chairs Aphrodite Galata (University of Manchester, UK) Branislav Kisaˇcanin (Delphi Corporation, USA) Program Committee K. Aizawa A. Hanjalic Q. Tian A. del Bimbo A. Jaimes M. Turk N. Boujemaa A. Kapoor J. Vitria I. Cohen M. Nixon G. Xu J. Cohn M. Pantic M. Yang J. Crowley I. Patras X. Zhou D. Gatica-Perez A. Pentland T. Gevers S. Sclaroff Corporate Sponsor Delphi Corporation, USA www.delphi.com Table of Contents Computer Vision in Human-Computer Interaction Robust Face Alignment Based on HierarchicalClassifier Network Li Zhang, Haizhou Ai, Shihong Lao .............................. 1 EigenExpress Approach in Recognition of Facial ExpressionUsing GPU Qi Wu, Mingli Song, Jiajun Bu, Chun Chen ...................... 12 Face Representation Method Using Pixel-to-Vertex Map (PVM) for 3D Model Based Face Recognition Taehwa Hong, Hagbae Kim, Hyeonjoon Moon, Yongguk Kim, Jongweon Lee, Seungbin Moon................................... 21 Robust Head Tracking with Particles Based on Multiple Cues Fusion Yuan Li, Haizhou Ai, Chang Huang, Shihong Lao .................. 29 Vision-Based Interpretation of Hand Gestures for Remote Control of a Computer Mouse Antonis A. Argyros, Manolis I.A. Lourakis ........................ 40 Computing Emotion Awareness Through Facial Electromyography Egon L. van den Broek, Marleen H. Schut, Joyce H.D.M. Westerink, Jan van Herk, Kees Tuinenbreijer................................ 52 Silhouette-Based Method for Object Classification and Human Action Recognition in Video Yi˘githan Dedeo˘glu, B. U˘gur To¨reyin, U˘gur Gu¨du¨kbay, A. Enis C¸etin ................................................. 64 Voice Activity Detection Using Wavelet-Based Multiresolution Spectrum and Support Vector Machines and Audio Mixing Algorithm Wei Xue, Sidan Du, Chengzhi Fang, Yingxian Ye .................. 78 Action Recognition in BroadcastTennis Video Using Optical Flow and Support Vector Machine Guangyu Zhu, Changsheng Xu, Wen Gao, Qingming Huang ......... 89 FaceMouse: A Human-Computer Interface for Tetraplegic People Emanuele Perini, Simone Soria, Andrea Prati, Rita Cucchiara ...... 99 XII Table of Contents Object Retrieval by Query with Sensibility Based on the KANSEI-Vocabulary Scale Sunkyoung Baek, Myunggwon Hwang, Miyoung Cho, Chang Choi, Pankoo Kim .................................................. 109 Author Index................................................... 121 Robust Face Alignment Based on Hierarchical Classifier Network Li Zhang1, Haizhou Ai1, and Shihong Lao2 1 Department of Computer Science, Tsinghua University, Beijing 100084, China 2 Sensingand Control Technology Lab,Omron Corporation, Kyoto 619-0283, Japan [email protected] Abstract. Robustfacealignmentiscrucialformanyfaceprocessingap- plications.Asfacedetectiononlygivesaroughestimationoffaceregion, one important problem is how to align facial shapes starting from this rough estimation, especially on face images with expression and pose changes. We propose a novel method of face alignment by building a hierarchical classifier network, connecting face detection and face align- ment into a smooth coarse-to-fine procedure. Classifiers are trained to recognize feature textures in different scales from entire face to local patterns. A multi-layer structure is employed to organize theclassifiers, which begins with one classifier at the first layer and gradually refines thelocalization offeaturepointsbymoreclassifiers inthefollowing lay- ers. A Bayesian framework is configured for the inference of the feature points between the layers. The boosted classifiers detects facial features discriminately from its local neighborhood, while the inference between the layers constrains the searching space. Extensive experiments are re- ported toshow its accuracy and robustness. 1 Introduction Face alignment, whose objective is to localize the feature points on face im- ages such as the contour points of eyes, noses, mouths and outlines, plays a fundamental role in many face processing tasks. The shape and texture of the feature points acquired by the alignment provide very helpful information for applications such as face recognition, modeling and synthesis. However, since the shape of the face may vary largely in practical images due to differences in age, expression and etc, a robust alignment algorithm, especially against errant initialization and face shape variation, is still a goal to achieve. There have been many studies on face alignment in the recent decade, most of which were based on Active Shape Model (ASM) and Active Appearance Model (AAM), proposed by Cootes et al [1]. In all these improvements,local or global texture features are employed to guide an iterative optimization of label points under the constraint of a statistical shape model. Many different types of features such as Gabor[2], Haar wavelet[3], and machine learning methods such as Ada-Boosting[4,5], k-NN[6] have been employedto replace the gradient T.S.Huangetal.(Eds.):HCI/ECCV2006,LNCS3979,pp.1–11,2006. (cid:1)c Springer-VerlagBerlinHeidelberg2006