Detecting Activities of Daily Living in First-person Camera Views Hamed Pirsiavash, Deva Ramanan Computer Science Department, UC Irvine 1 Motivation A sample video of Activities of Daily Living 2 Applications Tele-rehabilitation Long-term at-home monitoring •  Kopp et al,, Arch. of Physical Medicine and Rehabilitation. 1997. •  Catz et al, Spinal Cord 1997. 3 Applications Life-logging So far, mostly “write-only” memory! This is the right time for computer vision community to get involved. •  Gemmell et al, “MyLifeBits: a personal database for everything.” Communications of the ACM 2006. •  Hodges et al, “SenseCam: A retrospective memory aid”, UbiComp, 2006. 4 Related work: action recognition There are quite a few video benchmarks for action recognition. UCF sports, CVPR’08 KTH, ICPR’04 Olympics sport, BMVC’10 Hollywood, CVPR’09 UCF Youtube, CVPR’08 VIRAT, CVPR’11 Collecting interesting but natural video is surprisingly hard. It is difficult to define action categories outside “sports” domain 5 Wearable ADL detection It is easy to collect natural data 6 Wearable ADL detection It is easy to collect ADL actions derived from medical natural data literature on patient rehabilitation 7 Outline •  Challenges –  What features to use? –  Appearance model –  Temporal model •  Our model –  “Active” vs “passive” objects –  Temporal pyramid •  Dataset •  Experiments 8 Challenges What features to use? Low level features High level features (Weak semantics) (Strong semantics) Human pose Space-time interest points Laptev, IJCV’05 Difficulties of pose: •  Detectors are not accurate enough •  Not useful in first person camera views 9 Challenges What features to use? Low level features High level features (Weak semantics) (Strong semantics) Space-time interest points Human pose Object-centric features Laptev, IJCV’05 Difficulties of pose: •  Detectors are not accurate enough •  Not useful in first person camera views 10