ebook img

Machine Learning in Computer Vision PDF

249 Pages·2005·6.514 MB·English
Save to my drive
Quick download
Download
Most books are stored in the elastic cloud where traffic is expensive. For this reason, we have a limit on daily download.

Preview Machine Learning in Computer Vision

Machine Learning in Computer Vision by N. SEBE University of Amsterdam, The Netherlands IRACOHEN HPResearch Labs, U.S.A. ASHUTOSH GARG Google Inc., U.S.A. and THOMAS S. HUANG University of Illinois at Urbana-Champaign, Urbana, IL, U.S.A. AC.I.P. Catalogue record for this book is available from the Library of Congress. ISBN-10 1-4020-3274-9 (HB) Springer Dordrecht, Berlin, Heidelberg, New York ISBN-10 1-4020-3275-7 (e-book) Springer Dordrecht, Berlin, Heidelberg, New York ISBN-13 978-1-4020-3274-5 (HB) Springer Dordrecht, Berlin, Heidelberg, New York ISBN-13 978-1-4020-3275-2 (e-book) Springer Dordrecht, Berlin, Heidelberg, New York Published by Springer, P.O. Box 17, 3300 AADordrecht, The Netherlands. Printed on acid-free paper All Rights Reserved © 2005 Springer No part of this work may be reproduced, stored in a retrieval system, or transmitted in any form or by any means, electronic, mechanical, photocopying, microfilming, recordingor otherwise, without written permission from the Publisher, with the exceptionof any material supplied specifically for the purpose of being entered and executed on a computer system, for exclusive use by the purchaser of the work. Printed in the Netherlands. Tomyparents Nicu ToMeravandYonatan Ira Tomyparents Asutosh Tomystudents: Past,present,andfuture Tom Contents Foreword xi Preface xiii 1. INTRODUCTION 1 1 ResearchIssuesonLearninginComputerVision 2 2 OverviewoftheBook 6 3 Contributions 12 2. THEORY: PROBABILISTICCLASSIFIERS 15 1 Introduction 15 2 PreliminariesandNotations 18 2.1 MaximumLikelihoodClassification 18 2.2 InformationTheory 19 2.3 Inequalities 20 3 BayesOptimalErrorandEntropy 20 4 Analysis of Classification Error of Estimated (Mismatched) Distribution 27 4.1 HypothesisTestingFramework 28 4.2 ClassificationFramework 30 5 DensityofDistributions 31 5.1 DistributionalDensity 33 5.2 RelatingtoClassificationError 37 6 ComplexProbabilisticModelsandSmallSampleEffects 40 7 Summary 41 vi MACHINELEARNINGINCOMPUTERVISION 3. THEORY: GENERALIZATIONBOUNDS 45 1 Introduction 45 2 Preliminaries 47 3 AMarginDistributionBasedBound 49 3.1 ProvingtheMarginDistributionBound 49 4 Analysis 57 4.1 ComparisonwithExistingBounds 59 5 Summary 64 4. THEORY: SEMI-SUPERVISEDLEARNING 65 1 Introduction 65 2 PropertiesofClassification 67 3 ExistingLiterature 68 4 Semi-supervised Learning Using Maximum Likelihood Estimation 70 5 Asymptotic Properties of Maximum Likelihood Estimation withLabeledandUnlabeledData 73 5.1 ModelIsCorrect 76 5.2 ModelIsIncorrect 77 5.3 Examples: Unlabeled Data Degrading Performance withDiscreteandContinuousVariables 80 5.4 GeneratingExamples: PerformanceDegradationwith UnivariateDistributions 83 5.5 DistributionofAsymptoticClassificationErrorBias 86 5.6 ShortSummary 88 6 LearningwithFiniteData 90 6.1 ExperimentswithArtificialData 91 6.2 Can Unlabeled Data Help with Incorrect Models? Bias vs. Variance Effects and the Labeled-unlabeled Graphs 92 6.3 Detecting When Unlabeled Data Do Not Change the Estimates 97 6.4 Using Unlabeled Data to Detect Incorrect Modeling Assumptions 99 7 ConcludingRemarks 100 Contents vii 5. ALGORITHM: MAXIMUMLIKELIHOODMINIMUMENTROPYHMM 103 1 PreviousWork 103 2 Mutual Information, Bayes Optimal Error, Entropy, and ConditionalProbability 105 3 MaximumMutualInformationHMMs 107 3.1 DiscreteMaximumMutualInformationHMMs 108 3.2 ContinuousMaximumMutualInformationHMMs 110 3.3 UnsupervisedCase 111 4 Discussion 111 4.1 Convexity 111 4.2 Convergence 112 4.3 Maximum A-posteriori View of Maximum Mutual InformationHMMs 112 5 ExperimentalResults 115 5.1 SyntheticDiscreteSupervisedData 115 5.2 SpeakerDetection 115 5.3 ProteinData 117 5.4 Real-timeEmotionData 117 6 Summary 117 6. ALGORITHM: MARGINDISTRIBUTIONOPTIMIZATION 119 1 Introduction 119 2 AMarginDistributionBasedBound 120 3 ExistingLearningAlgorithms 121 4 TheMarginDistributionOptimization(MDO)Algorithm 125 4.1 ComparisonwithSVMandBoosting 126 4.2 ComputationalIssues 126 5 ExperimentalEvaluation 127 6 Conclusions 128 7. ALGORITHM: LEARNING THE STRUCTURE OF BAYESIAN NETWORKCLASSIFIERS 129 1 Introduction 129 2 BayesianNetworkClassifiers 130 2.1 NaiveBayesClassifiers 132 2.2 Tree-AugmentedNaiveBayesClassifiers 133 viii MACHINELEARNINGINCOMPUTERVISION 3 SwitchingbetweenModels: NaiveBayesandTANClassifiers 138 4 Learning the Structure of Bayesian Network Classifiers: ExistingApproaches 140 4.1 Independence-basedMethods 140 4.2 LikelihoodandBayesianScore-basedMethods 142 5 ClassificationDrivenStochasticStructureSearch 143 5.1 StochasticStructureSearchAlgorithm 143 5.2 Adding VC Bound Factor to the Empirical Error Measure 145 6 Experiments 146 6.1 ResultswithLabeledData 146 6.2 ResultswithLabeledandUnlabeledData 147 7 ShouldUnlabeledDataBeWeighedDifferently? 150 8 ActiveLearning 151 9 ConcludingRemarks 153 8. APPLICATION: OFFICEACTIVITYRECOGNITION 157 1 Context-SensitiveSystems 157 2 TowardsTractableandRobustContextSensing 159 3 LayeredHiddenMarkovModels(LHMMs) 160 3.1 Approaches 161 3.2 DecompositionperTemporalGranularity 162 4 ImplementationofSEER 164 4.1 FeatureExtractionandSelectioninSEER 164 4.2 ArchitectureofSEER 165 4.3 LearninginSEER 166 4.4 ClassificationinSEER 166 5 Experiments 166 5.1 Discussion 169 6 RelatedRepresentations 170 7 Summary 172 9. APPLICATION: MULTIMODALEVENTDETECTION 175 1 FusionModels: AReview 176 2 AHierarchicalFusionModel 177 2.1 WorkingoftheModel 178 2.2 TheDurationDependentInputOutputMarkovModel 179 Contents ix 3 ExperimentalSetup,Features,andResults 182 4 Summary 183 10.APPLICATION: FACIALEXPRESSIONRECOGNITION 187 1 Introduction 187 2 HumanEmotionResearch 189 2.1 AffectiveHuman-computerInteraction 189 2.2 TheoriesofEmotion 190 2.3 FacialExpressionRecognitionStudies 192 3 FacialExpressionRecognitionSystem 197 3.1 FaceTrackingandFeatureExtraction 197 3.2 BayesianNetworkClassifiers: Learningthe “Structure”oftheFacialFeatures 200 4 ExperimentalAnalysis 201 4.1 ExperimentalResultswithLabeledData 204 4.1.1 Person-dependentTests 205 4.1.2 Person-independentTests 206 4.2 ExperimentswithLabeledandUnlabeledData 207 5 Discussion 208 11.APPLICATION: BAYESIANNETWORKCLASSIFIERSFORFACEDETECTION 211 1 Introduction 211 2 RelatedWork 213 3 ApplyingBayesianNetworkClassifierstoFaceDetection 217 4 Experiments 218 5 Discussion 222 References 225 Index 237 Foreword It started with image processing in the sixties. Back then, it took ages to digitizeaLandsatimageandthenprocessitwithamainframecomputer. Pro- cessing was inspired on the achievements of signal processing and was still verymuchorientedtowardsprogramming. In the seventies, image analysis spun off combining image measurement with statistical pattern recognition. Slowly, computational methods detached themselvesfromthesensorandthegoaltobecomemoregenerallyapplicable. In the eighties, model-driven computer vision originated when artificial in- telligenceandgeometricmodellingcametogetherwithimageanalysiscompo- nents. The emphasis was on precise analysis with little or no interaction, still very much an art evaluated by visual appeal. The main bottleneck was in the amountofdatausinganaverageof5to50picturestoillustratethepoint. At the beginning of the nineties, vision became available to many with the advent of sufficiently fast PCs. The Internet revealed the interest of the gen- eral public im images, eventually introducing content-based image retrieval. Combining independent (informal) archives, as the web is, urges for interac- tive evaluation of approximate results and hence weak algorithms and their combinationinweakclassifiers. In the new century, the last analog bastion was taken. In a few years, sen- sors have become all digital. Archives will soon follow. As a consequence ofthischangeinthebasicconditionsdatasetswilloverflow. Computervision will spin off a new branch to be called something like archive-based or se- manticvisionincludingaroleforformalknowledgedescriptioninanontology equippedwithdetectors. Analternativeviewisexperience-based orcognitive vision. Thisismostlyadata-drivenviewonvisionandincludestheelementary lawsofimageformation. Thisbookcomesrightontime. Thegeneraltrendiseasytosee. Themeth- odsofcomputationwentfromdedicatedtoonespecifictasktomoregenerally applicable building blocks, from detailed attention to one aspect like filtering

See more

The list of books you might like

Most books are stored in the elastic cloud where traffic is expensive. For this reason, we have a limit on daily download.