NONLINEAR ESTIMATION FOR VISION-BASED AIR-TO-AIR TRACKING A Thesis Presented to The Academic Faculty by Seung-Min Oh In Partial Fulfillment of the Requirements for the Degree Doctor of Philosophy in the School of Aerospace Engineering Georgia Institute of Technology December 2007 NONLINEAR ESTIMATION FOR VISION-BASED AIR-TO-AIR TRACKING Approved by: Professor Eric N. Johnson, Professor Allen R. Tannenbaum Committee Chair School of Electrical and Computer School of Aerospace Engineering Engineering Georgia Institute of Technology Georgia Institute of Technology Professor Anthony J. Calise Professor George Vachtsevanos School of Aerospace Engineering School of Electrical and Computer Georgia Institute of Technology Engineering Georgia Institute of Technology Professor Eric Feron Date Approved: 7 November 2007 School of Aerospace Engineering Georgia Institute of Technology To my wife, Miyoung, and our parents, our three daughters, Soobin, Jiyoon, and Chaeyoung, for their love, patience, and encouragement. iii ACKNOWLEDGEMENTS Above all, I would like to express my sincere gratitude to my advisor, Professor Eric N. Johnson, who first opened my eyes to nonlinear filters and advised this work, for his support, guidance, and timely advice. I learned a lot from him in all respects of education and research. This dissertation would not have been at this point without his guidance. I would like to thank my thesis committee, Dr. Anthony J. Calise, Dr. Eric Feron, Dr. Allen R. Tannenbaum, and Dr. George Vachtsevanos for sharing their precious time and comments. I am honored to have them all in my prestigious committee members. My additional special thanks go to Dr. Calise who thoroughly read this dissertation and gave very detailed comments until the last moment even though he couldn’t attend my defense presentation. I would like to offer my thanks to my lab mates, Jincheol Ha and Yoko Watanabe. Spending the many years in the same lab, they helped me get out of my ignorance in many areas through precious discussions. I would also like to express my appreciation toAllenWuandDr. RamachandraSattigeriforsharingtheirprecioustimeandefforts to proof-reading and discussions. I am lucky to be with wonderful members of UAV Lab, past and present, Dr. Shresh K. Kannan, Dr. Jim Neidhoefer, Nimrod Rooz, Girish Chowdhary, Claus Christmann,JenniferSheffield,SyedShah,JeongHur,WaynePickell,HenrikChristo- phersen, Alison Proctor, Gregory Ivey, Shannon Twigg, Stewart Geyer, Michael Turbe, Sumit Mishra, Tom Apker, Mike Curry. They all helped me at some points or through invaluable memories and discussions. Iamalsoindebtedtoothercontrolgroupmembers, Dr. Bong-JunYang, Dongwon iv Jung, and my others for their help in various ways throughout my stay at Georgia Tech. IwouldalsoliketothankDrs. AliTurkerKutay, ChangChen, YoonghyunShin for their help during their stay at Georgia Tech as former control group members. Finally, I would like to thank my family. I could have felt much more difficulty without their love and encouragement. I would like to thank my parents for their love, support, and encouragement throughout my life. I express my deepest love and appreciation to my special four ladies: my foremost special lady, my wife Miyoung, and my three daughters Soobin, Jiyoon, and Chaeyoung, thank you for your support, encouragement, patience, sacrifice, and love. Together with you all on this rather long journey, this has been more lively, fun, encouraging, and meaningful. You kept me always on the right track. v TABLE OF CONTENTS DEDICATION . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . iii ACKNOWLEDGEMENTS . . . . . . . . . . . . . . . . . . . . . . . . . . . . iv LIST OF TABLES . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . x LIST OF FIGURES . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xi LIST OF SYMBOLS OR ABBREVIATIONS . . . . . . . . . . . . . . . . . . xiv SUMMARY . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xv 1 BACKGROUND . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1 1.1 Motivation and Research Outlines . . . . . . . . . . . . . . . . . . 3 1.2 Nonlinear Estimation . . . . . . . . . . . . . . . . . . . . . . . . . . 8 1.2.1 Extended Kalman Filter (EKF) and Other Nonlinear Filters 9 1.2.2 Unscented Kalman Filters and Derivative-Free Filters . . . . 11 1.2.3 Sequential Monte Carlo Estimation Methods (Particle Filters) 15 1.3 Integrated Navigation System . . . . . . . . . . . . . . . . . . . . . 18 1.4 Nonlinear Filtering and Applications to Target Tracking Problems . 23 1.4.1 Bearings-Only Tracking . . . . . . . . . . . . . . . . . . . . 23 1.4.2 Maneuvering Target Tracking . . . . . . . . . . . . . . . . . 25 1.5 Vision-Based Tracking System . . . . . . . . . . . . . . . . . . . . . 26 1.6 The Outline of Thesis . . . . . . . . . . . . . . . . . . . . . . . . . 28 2 NONLINEAR ESTIMATION . . . . . . . . . . . . . . . . . . . . . . . . 29 2.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29 2.2 Mathematical Preliminaries . . . . . . . . . . . . . . . . . . . . . . 30 2.3 Extended Kalman Filters . . . . . . . . . . . . . . . . . . . . . . . 37 2.3.1 Standard Extended Kalman Filter . . . . . . . . . . . . . . 38 2.3.2 ExtendedKalmanFilterwithSequentialMeasurementUpdates 40 2.4 Unscented Kalman Filters . . . . . . . . . . . . . . . . . . . . . . . 42 vi 2.4.1 Unscented Transformation . . . . . . . . . . . . . . . . . . . 42 2.4.2 Accuracy Analysis of Unscented Transformation . . . . . . . 45 2.4.3 Standard Unscented Kalman Filter (UKF) . . . . . . . . . 51 2.4.4 UnscentedKalmanFilterwithSequentialMeasurementUpdate 54 2.5 Sequential Monte Carlo Methods (Particle Filters) . . . . . . . . . 56 2.5.1 Perfect Monte Carlo Sampling . . . . . . . . . . . . . . . . 58 2.5.2 Bayesian Importance Sampling . . . . . . . . . . . . . . . . 59 2.6 Extended Marginalized Particle Filter . . . . . . . . . . . . . . . . 67 2.6.1 Marginalization or Rao Blackwellization . . . . . . . . . . . 68 2.6.2 Extended Marginalized Particle Filter (EMPF) . . . . . . . 70 3 INTEGRATED NAVIGATION SYSTEM . . . . . . . . . . . . . . . . . 74 3.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 74 3.2 Description of Inertial Navigation System . . . . . . . . . . . . . . 76 3.2.1 Continuous Process Model of INS Navigation . . . . . . . . 79 3.2.2 Discrete Process Model of INS Navigation . . . . . . . . . . 84 3.3 INS Navigation Measurement Model . . . . . . . . . . . . . . . . . 86 3.3.1 DGPS Position and Velocity Measurement Model . . . . . . 89 3.3.2 Magnetometer Measurement Model . . . . . . . . . . . . . . 90 3.3.3 Quaternion Norm Pseudo-Measurement Model . . . . . . . 91 3.4 Integrated INS Navigation Using EKF and UKF . . . . . . . . . . 92 3.4.1 ExtendedKalmanFilterwithSequentialMeasurementUpdates 93 3.4.2 Unscented Kalman Filter with Sequential Measurement Up- dates . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 98 3.5 Simulation Model and Filter Performance Simulation . . . . . . . . 100 3.5.1 Simulation Model . . . . . . . . . . . . . . . . . . . . . . . . 101 3.5.2 Filter Performance Simulation . . . . . . . . . . . . . . . . . 104 3.6 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 114 vii 4 VISION-BASEDTRACKINGSYSTEMBASEDONUNSCENTEDKALMAN FILTER . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 116 4.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 116 4.2 System Description for Vision-Based Relative Navigation . . . . . . 120 4.2.1 Process Model of Vision-based Relative Navigation . . . . . 121 4.2.2 Measurement Model of Vision-based Relative Navigation . . 126 4.2.3 Measurement from the Image Processor . . . . . . . . . . . 127 4.3 Simulation and Results . . . . . . . . . . . . . . . . . . . . . . . . . 128 4.3.1 Initialization and Noise Covariance Setting . . . . . . . . . . 128 4.3.2 Simulation Results . . . . . . . . . . . . . . . . . . . . . . . 129 4.4 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 136 5 VISION-BASEDTRACKINGSYSTEMBASEDONEXTENDEDMARGINAL- IZED PARTICLE FILTER . . . . . . . . . . . . . . . . . . . . . . . . . 138 5.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 138 5.2 System Description for Vision-Based Relative Navigation . . . . . . 140 5.2.1 General State Space Formulation for Vision-Based Tracking 141 5.2.2 Measurement Model of Vision-based Tracking System . . . . 143 5.2.3 Measurement from the Image Processor . . . . . . . . . . . 144 5.2.4 Marginalization or Rao Blackwellization . . . . . . . . . . . 145 5.3 Simulation and Results . . . . . . . . . . . . . . . . . . . . . . . . . 149 5.3.1 Initialization and Noise Covariance Setting . . . . . . . . . . 149 5.3.2 Simulation Results . . . . . . . . . . . . . . . . . . . . . . . 150 5.4 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 156 6 CONCLUSIONS AND RECOMMENDED FUTURE RESEARCH . . . 158 6.1 Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 158 6.2 Recommended Future Research . . . . . . . . . . . . . . . . . . . . 162 APPENDIX A EKF FORMULATION WITH SEQUENTIAL MEASURE- MENT UPDATE . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 164 REFERENCES . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 169 viii VITA . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 180 ix LIST OF TABLES 3.1 Sensor Update Rates . . . . . . . . . . . . . . . . . . . . . . . . . . . 87 x