Advances in Computer Vision and Pattern Recognition Shaogang Gong Marco Cristani Shuicheng Yan Chen Change Loy Editors Person Re-Identification Advances in Computer Vision and Pattern Recognition For further volumes: http://www.springer.com/series/4205 Shaogang Gong • Marco Cristani Shuicheng Yan • Chen Change Loy Editors Person Re-Identification 123 Editors Shaogang Gong Shuicheng Yan Queen Mary University National University of Singapore London Singapore UK Chen Change Loy Marco Cristani The Chinese University of Hong Kong University of Verona Shatin Verona Hong Kong SAR Italy Series editors Sameer Singh Sing Bing Kang Rail Vision Europe Ltd. Interactive Visual Media Group Castle Donington Microsoft Research Leicestershire, UK Redmond, WA, USA ISSN 2191-6586 ISSN 2191-6594 (electronic) Advances in Computer Vision and Pattern Recognition ISBN 978-1-4471-6295-7 ISBN 978-1-4471-6296-4 (eBook) DOI 10.1007/978-1-4471-6296-4 Springer London Heidelberg New York Dordrecht Library of Congress Control Number: 2013957125 Ó Springer-Verlag London 2014 This work is subject to copyright. All rights are reserved by the Publisher, whether the whole or part of the material is concerned, specifically the rights of translation, reprinting, reuse of illustrations, recitation, broadcasting, reproduction on microfilms or in any other physical way, and transmission or information storage and retrieval, electronic adaptation, computer software, or by similar or dissimilar methodology now known or hereafter developed. Exempted from this legal reservation are brief excerpts in connection with reviews or scholarly analysis or material supplied specifically for the purpose of being entered and executed on a computer system, for exclusive use by the purchaser of the work. Duplication of this publication or parts thereof is permitted only under the provisions of the Copyright Law of the Publisher’s location, in its current version, and permission for use must always be obtained from Springer. Permissions for use may be obtained through RightsLink at the Copyright Clearance Center. Violations are liable to prosecution under the respective Copyright Law. The use of general descriptive names, registered names, trademarks, service marks, etc. in this publication does not imply, even in the absence of a specific statement, that such names are exempt from the relevant protective laws and regulations and therefore free for general use. While the advice and information in this book are believed to be true and accurate at the date of publication, neither the authors nor the editors nor the publisher can accept any legal responsibility for any errors or omissions that may be made. The publisher makes no warranty, express or implied, with respect to the material contained herein. Printed on acid-free paper Springer is part of Springer Science+Business Media (www.springer.com) Preface Person re-identification is the problem of recognising and associating a person at different physical locations over time after the person had been previously observed visually elsewhere. Solving the re-identification problem has gained a rapid increase in attention in both academic research communities and industrial laboratories in recent years. The problem has many manifestations from different application domains. For instance, the problem is known as ‘‘re-acquisition’’ when the aim is to associate a target (person) when it is temporarily occluded during the tracking in a single camera view. On the other hand, in domotics applications or personalised healthcare environments, the primary aim is to retain the identity of a person whilst one is moving about in a private home of distributed spaces, e.g. crossing multiple rooms. Re-identification can provide a useful tool for validating the identity of impaired or elderly people in a seamless way without the need for more invasive biometric verification procedures, e.g. controlled face or fingerprint recognition. Moreover, in a human–robot interaction scenario, solving the re-identification problem can be considered as ‘‘non-cooperative target recognition’’, where the identity of the interlocutor is maintained, allowing the robot to be continuously aware of the surrounding people. In larger distributed spaces such as airport ter- minals and shopping malls, re-identification is mostly considered as the task of ‘‘object association’’ in a distributed multi-camera network, where the goal is to keep track of an individual across different cameras with non-overlapping field of views. For instance, in a multi-camera surveillance system, re-identification is needed to trace the inter-camera whereabouts of individuals of interest (a watch-list), or simply to understand how people move in complex environments such as an airport and a train station for better crowd traffic management and crowding control. In a retail environment, re-identification can provide useful information for improving cus- tomer service and shopping space management. In a more general setting of online shopping, re-identification of visual objects of different categories, e.g. clothing, can help in tagging automatically huge volumes of visual samples of consumer goods in Internet image indexing, search and retrieval. Solving the person re-identification problem poses a considerable challenge that requires visually detecting and recognising a person (subject) at different space time locations observed under substantially different, and often unknown, viewing conditions without subject collaboration. Early published work on re-identification can date back a decade ago to 2003, but most contemporary techniques have been v vi Preface developed since 2008, and in particular in the last 2–3 years. In the past 5 years, there has been a tremendous increase in computer vision research on solving the re-identification problem, evident from a large number of academic papers pub- lished in all the major conferences (ICCV, CVPR, ECCV, BMVC, ICIP) and journals (TPAMI, IJCV, Pattern Recognition). This trend will increase further in the coming years, given that many open problems remain unsolved. Inspired by the First International Workshop on Re-Identification held at Florence in Italy in October 2012, this book is a collection of invited chapters from 0 some of the world s leading researchers working on solving the re-identification problem. It aims to provide a comprehensive and in-depth presentation of recent progress and the current state-of-the-art approaches to solving some of the fun- damental challenges in person re-identification, benefiting from wider research in the computer vision, pattern recognition and machine learning communities, and drawing insights from video analytics system design considerations for engi- neering practical solutions. Due to its diverse nature, the development of person re- identification methods by visual matching has been reported in a wide range of fields, from multimedia to robotics, from domotics to visual surveillance, but all with an underlying computer vision theme. Re-identification exploits extensively many core computer vision techniques that aim at extracting and representing an individual’s visual appearance in a scene, e.g. pedestrian detection and tracking, and object representation; and machine learning techniques for discriminative matching, e.g. distance metric learning and transfer learning. Moreover, solving the person re-identification problem can benefit from exploiting heterogeneous information by learning more effective semantic attributes, exploiting spatio- temporal statistics, estimating feature transformation across different cameras, taking into account soft-biometric cues (e.g. height, gender) and considering contextual cues (e.g. baggage, other people nearby). This book is the first dedicated treatment on the subject of Person Re-Identi- fication that aims to address a highly focused problem with a strong multidisci- plinary appeal to practitioners in both fundamental research and practical applications. In the context of video content analysis, visual surveillance and human recognition, there are a number of other books published recently that aim to address a wider range of topics, e.g. Video Analytics for Business Intelligence, by Caifeng Shan, Fatih Porikli, Tao Xiang and Shaogang Gong (2012); Visual Analysis of Behaviour: From Pixels to Semantics, by Shaogang Gong and Tao Xiang (2011); and Visual Analysis of Humans: Looking at People, by Thomas Moeslund, Adrian Hilton, Volker Kruger and Leonid Sigal (2011). In contrast to those other books, this book provides a more in-depth analysis and a more com- prehensive presentation of techniques required specifically for solving the problem of person re-identification. Despite aiming to address a highly focused problem, the techniques presented in this book, e.g. feature representation, attribute learn- ing, ranking, active learning and transfer learning, are highly applicable to other more general problems in computer vision, pattern recognition and machine learning. Therefore, the book should also be of considerable interest to a wider audience. Preface vii We anticipate that this book will be of special interest to academics, post- graduates and industrial researchers specialised in computer vision and machine learning, database (including internet) image retrieval, big data mining and search engines. It should also be of interest to commercial developers and managers keen to exploit this emerging technology for a host of applications including security and surveillance, personalised healthcare, commercial information profiling, business intelligence gathering, smart city, public space infrastructure manage- ment, consumer electronics and retails. Finally, this book will also be of use to postgraduate students of computer science, engineering, applied mathematics and statistics, cognitive and social studies. London, October 2013 Shaogang Gong Verona Marco Cristani Singapore Shuicheng Yan Hong Kong Chen Change Loy Acknowledgments The preparation of this book has required the dedication of many people. First of all, we thank all the contributing authors for their extraordinary effort and dedi- cation in preparing the book chapters within a very tight time frame. Second, we express our gratitude to all the reviewers. Their critical and constructive feedback helped in improving the quality of the book. Finally, we thank Simon Rees and Wayne Wheeler at Springer for their support throughout the preparation of this book. The book was typeset using LaTeX. This book was inspired by the First International Workshop on Re-Identifica- tion (Re-Id 2012), in conjunction with the European Conference on Computer Vision, held at Florence in Italy in October 2012. To that end, we thank the workshop programme committee and the authors who made the workshop a huge success. We also thank the workshop industrial sponsors Bosch, KAI Square, Vision Semantics and Embedded Vision Systems who sponsored the Best Paper Award prize and made the workshop a more rewarding experience. ix Contents 1 The Re-identification Challenge. . . . . . . . . . . . . . . . . . . . . . . . . . 1 Shaogang Gong, Marco Cristani, Chen Change Loy and Timothy M. Hospedales Part I Features and Representations 2 Discriminative Image Descriptors for Person Re-identification . . . 23 Bingpeng Ma, Yu Su and Frédéric Jurie 3 SDALF: Modeling Human Appearance with Symmetry-Driven Accumulation of Local Features . . . . . . . . . . . . . . . . . . . . . . . . . 43 Loris Bazzani, Marco Cristani and Vittorio Murino 4 Re-identification by Covariance Descriptors . . . . . . . . . . . . . . . . 71 Sławomir Ba˛k and François Brémond 5 Attributes-Based Re-identification . . . . . . . . . . . . . . . . . . . . . . . . 93 Ryan Layne, Timothy M. Hospedales and Shaogang Gong 6 Person Re-identification by Attribute-Assisted Clothes Appearance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 119 Annan Li, Luoqi Liu and Shuicheng Yan 7 Person Re-identification by Articulated Appearance Matching . . . 139 Dong Seon Cheng and Marco Cristani 8 One-Shot Person Re-identification with a Consumer Depth Camera . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 161 Matteo Munaro, Andrea Fossati, Alberto Basso, Emanuele Menegatti and Luc Van Gool xi xii Contents 9 Group Association: Assisting Re-identification by Visual Context. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 183 Wei-Shi Zheng, Shaogang Gong and Tao Xiang 10 Evaluating Feature Importance for Re-identification . . . . . . . . . . 203 Chunxiao Liu, Shaogang Gong, Chen Change Loy and Xinggang Lin Part II Matching and Distance Metric 11 Learning Appearance Transfer for Person Re-identification. . . . . 231 Tamar Avraham and Michael Lindenbaum 12 Mahalanobis Distance Learning for Person Re-identification . . . . 247 Peter M. Roth, Martin Hirzer, Martin Köstinger, Csaba Beleznai and Horst Bischof 13 Dictionary-Based Domain Adaptation Methods for the Re-identification of Faces. . . . . . . . . . . . . . . . . . . . . . . . . 269 Qiang Qiu, Jie Ni and Rama Chellappa 14 From Re-identification to Identity Inference: Labeling Consistency by Local Similarity Constraints . . . . . . . . . 287 Svebor Karaman, Giuseppe Lisanti, Andrew D. Bagdanov and Alberto Del Bimbo 15 Re-identification for Improved People Tracking . . . . . . . . . . . . . 309 François Fleuret, Horesh Ben Shitrit and Pascal Fua Part III Evaluation and Application 16 Benchmarking for Person Re-identification . . . . . . . . . . . . . . . . . 333 Roberto Vezzani and Rita Cucchiara 17 Person Re-identification: System Design and Evaluation Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 351 Xiaogang Wang and Rui Zhao 18 People Search with Textual Queries About Clothing Appearance Attributes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 371 Riccardo Satta, Federico Pala, Giorgio Fumera and Fabio Roli