ETH Library Unmanned Solar Airplanes Design and Algorithms for Efficient and Robust Autonomous Operation Doctoral Thesis Author(s): Leutenegger, Stefan Publication date: 2014 Permanent link: https://doi.org/10.3929/ethz-a-010255301 Rights / license: In Copyright - Non-Commercial Use Permitted This page was generated automatically upon download from the ETH Zurich Research Collection. For more information, please consult the Terms of use. DISS. ETH NO. 22113 UNMANNED SOLAR AIRPLANES Design and Algorithms for Efficient and Robust Autonomous Operation A thesis submitted to attain the degree of DOCTOR OF SCIENCES of ETH ZURICH (Dr. sc. ETH Zurich) presented by STEFAN LEUTENEGGER MSc. Mechanical Engineering with Focus Robotics, ETH Zurich BSc. Mechanical Engineering, ETH Zurich born on 16.02.1983 citizen of Switzerland and Slovenia accepted on the recommendation of Prof. Roland Y. Siegwart (ETH Zurich, Principal Advisor) Prof. Gerd Hirzinger (DLR, Member of the Jury) Dr. Kurt Konolige (Industrial Perception, Member of the Jury) 2014 Autonomous Systems Lab ETH Zurich Switzerland c 2014 Stefan Leutenegger. All rights reserved. (cid:13) To my family and friends. Acknowledgements I would like to take the chance of expressing my deepest appreciation and gratitude to Prof. Roland Siegwart, who offered me the unique opportunity of pursuing my doctorate in the Autonomous Systems Lab (ASL): Roland, you have always been able to motivate me and provide valuable inputs while leaving me with the freedom to follow my own ideas. I would furthermore like to thank my postdocs Dr. Paul Furgale for his excellent guidance and ability to challenge me—as well as Dr. Margarita Chli who has been an inspiring adviser unconditionally supporting me also in rough times. You both have been great colleagues and friends! Moreover, many thanks go to Prof. Gerd Hirzinger for accepting to join the committee. Equally I would like to thank my second external committee member, Dr. Kurt Konolige, who has been supervising me since the time I spent at Willow Garage. Kurt, I will never forget the endless drive home from skiing with all its enriching discussions! I had the luck of interacting with many scientific and industrial partners in a fruitful manner. I would like to thank Dr. Reiner Kickert, Christian Ückert and Stephan Wrobel for working together on the senseSoar airplane. I am equally grateful for working with Sam Gisiger at Triadis on integrating glider avionic hardware and algorithms. Having had the ASL internal lead of the ICARUS and SHERPA FP7 project has given me the opportunity of interacting with many outstanding and visionary scientists, most notably with the project coordinators Dr. Geert de Cubber and Prof. Lorenzo Marconi, who will continue to be role models for me. During the past years, it was my pleasure to work with many students at Bachelor and Master level: Mathieu Jabas, Flavio Gohl, Philipp Krüsi, Atilla Yilmaz, Nikolas Gehrig, Fadri Furrer, Péter Fankhauser and Corsin Gwerder, Benjamin Peter, as well as all the focus team “Skye”, just to name a few of the many who dedicated their work to this fascinating field of research under my supervision. My colleagues have made every hour spent working in robotics enjoyable: many thanks go to the “fixed-wing” team consisting of Konrad Rudin, Thomas Mantel, Philipp Oettershagen, Amir Melzer, Dr. Kostas Alexis and our pilot and man for building and repairs, Rainer Lotz. It has been great to work with all of you and much of the progress reported in this thesis has been made thanks to this collaboration! The same holds for the remaining “flying” researchers, Dr. Stephan Weiss, Dr. Laurent Kneip, Dr. Markus Achtelik, Dr. Mike Bosse, Jörn Rehder, Janosch Nikolic, Sammy Omari, Michael Burri and Pascal Gohl. I am furthermore very thankful for the support by the lab engineers Thomas Baumgartner, Stefan Bertschi, as well as the lab administration by Lucy Borsatti and Cornelia della Casa; I am equally grateful for the help by Markus Bühler and Dario Fenner at the workshop, who I challenged repeatedly with parts difficult to manufacture. ASL has been a great work environment, and I do not want to miss thanking every iii individual for fruitful collaboration and also the fun times ranging from coffee breaks to lab trips and going out till the morning dawn. I would also like to express special thanks to Dr. Vincent Rabaud at Willow Garage, for mentoring equally as for the fun time spent with other colleagues like Ioan Sucan, Morten Kjægaard, Mihai Dolha, Stefan Holzer, and many more. My family and friends, to whom I dedicate this work (belonging to this special group of “normal people”), have unconditionally supported me at all times and thus deserve the highest gratitude. René and Bert, I will never forget our legendary 3 30 party! Dear × mom, dad and brother: thank you so much for having been there for me at all times! Zurich, in June 2014. iv Abstract Solar airplanes offer the unique capability of staying airborne for extremely long times: to date, both unmanned and manned systems have proved sustained flight, in the sense of flying through several day and night cycles. During the day, the solar module powers the airplane and re-charges a battery, which must take the airplane through the following night. Small-scale unmanned solar airplanes have thus been suggested for a plethora of non-military application scenarios, ranging from disaster response to Search and Rescue (SaR) as well as general large-scale mapping missions. This thesis addresses many aspects related to long-term autonomous operation of this special class of Unmanned Aerial Systems (UAS) in close proximity to the ground. We start in the very beginning with asking the question of how large a solar airplane should be and how it would perform, in order to accomplish a target mission. A methodo- logy is presented that performs actual aerodynamics and structural calculations of either a simplified shell or rib wing concept. The performance evaluation part also accounts for flying optimized altitude profiles, in order to allow for potential energy storage. The output of this conceptual design tool has motivated the design of the senseSoar solar airplane prototype that is equipped with enhanced sensing and processing components. We describe the details associated with the design of the different components—an engineering effort that spans various disciplines from aerodynamics to solar technology, electronics and avionics as well as structures. Furthermore, we introduce a modular sensing and processing unit that can be attached to a second solar airplane prototype, AtlantikSolar, aimed at record flying. The airplane was developed outside the scope of this work, but its realization was again motivated by the conceptual design tool. Throug- hout the design process, but also for subsequent simulations and autopilot development, aerodynamics and flight kinematics models play an important role; we present a complete toolchain for such analysis. Developing efficient components is key to any successful solar airplane design. Long- termoperation,however,willonlybeenabled,iftheaircraftadditionallyexhibitssufficient robustness. These two central concepts do not only apply to design, but equally to algo- rithms, which eventually turn the airplane into a system that can operate autonomously. As a basis for any autonomy, the aircraft needs to have an estimate about its internal states, as well as about its surroundings, specifically in the form of a map. Large parts of the thesis at hand address precisely the associated challenge of fusing various sensor sources under hard real-time and computational constraints. Specifically, two algorithms are presented and analyzed in detail that share a common element:namelyinertialmeasurements,i.e.accelerometerandrategyroreadingssubjected to their kinematics equations. A first fusion strategy complements this inertial module with magnetometer, static and dynamic pressure, as well as GPS measurements that are v combined in an Extended Kalman Filtering (EKF) framework. The approach comes with robustness as it can withstand long-term GPS outage. Respective results show how the crucial states of orientation and airspeed including Angle of Attack (AoA) and sideslip are still tracked sufficiently well in such a case. The latter is mainly enabled by the use of an aerodynamic airplane model. The algorithm was designed to be lightweight, so it can run our microcontroller boards as an input to the autopilot. A second estimation algorithm complements the inertial module with visual cues: the combination has gained increasing attention lately since it enables accurate state estimation as well as situational awareness. We chose an approach to fuse the landmark reprojection error with inertial terms in nonlinear optimization. The concept of keyframes is implemented by resorting to marginalization, i.e. partial linearization and variable elimination, in order to keep the optimization problem bounded—while it may still span a long time interval. The framework can be used with a monocular, stereo or multi- camera setup. As we demonstrate in extensive experiments, our algorithm outperforms a competitive filtering-based approach consistently in terms of accuracy, whilst admittedly demanding more computation. In further experiments, we address calibration of the camera pose(s) relative to the inertial measurement unit; and we validate the scalability of the method—which allows to size the optimization window to the available hardware— such that real-time constraints are respected. In order to address the specific challenge of forming visual keypoint associations, we propose BRISK: Binary Robust Invariant Scalable Keypoints that aims at providing a high-speed yet high quality alternative to proven algorithms like SIFT and SURF. The scheme consists of scale-space corner detection, keypoint orientation estimation and extraction of a binary descriptor string. The latter two steps employ a sampling pattern for local brightness gradient computation and descriptor assembly from brightness comparisons. The evaluation comprises detection repeatability and descriptor similarity as well as timings when compared to SIFT, SURF, BRIEF, and FREAK. The code has been released and has found broad adoption. The presented platforms with their sensing and processing capabilities are finally used in flight tests to run the suggested algorithms. We show online operation of the inertial navigation filter. Moreover, the stereo-visual-inertial fusion was run on-board a multicopter in the control loop, which has enabled a high level of autonomy. Finally, we demonstrate the application of the mono-visual-inertial algorithm to an AtlantikSolar flight. The algorithm was augmented to accept GPS and magnetometer measurements, outputting airplane state and a consistent map—as a first milestone towards autonomous operations of small solar airplanes close to the terrain. Keywords: Solar Airplanes, Unmanned Aerial Systems, Airplane Design, Sensor Fu- sion, State Estimation, Inertial Navigation System, Visual-Inertial Navigation System, Visual-Inertial Odometry, Simultaneous Localization and Mapping, Keyframes, Nonli- near Optimization, Image Keypoints, Image Features, Keypoint Detection, Descriptor Extraction. vi Kurzfassung SolarflugzeugebesitzendieeinzigartigeFähigkeit,überextremlangeZeiträumeinderLuft zu bleiben: heutzutage existieren sowohl bemannte als auch unbemannte solcher Systeme, welche bewiesen haben, dass sie kontinuierlich, also mehrere Tag- und Nacht-Zyklen unun- terbrochen fliegen können. Dabei versorgt ein Solarmodul während des Tages den Antrieb mit Leistung und lädt gleichzeitig einen Akku, welcher das Flugzeug dann durch die folgende Nacht bringen muss. Davon inspiriert wurden kleine unbemannte Solarflugzeuge für verschiedenste nicht-militärische Anwendungen vorgeschlagen; beispielsweise für den Einsatz in einem Katastrophengebiet, für Such- und Rettungs-Aufgaben oder generell für ausgedehnte Kartierungsanwendungen. Diese Dissertation behandelt mehrere Aspekte des Solarflugs kleiner unbemannter Flugzeugsysteme, welche nahe am Gelände operieren. Wir beginnen mit der Frage, wie gross denn ein Solarflugzeug auszulegen ist, um eine bestimte Mission erfüllen zu können, und welche Leistungscharakteristiken erwartet werden können. Es wird ein Ansatz vorgestellt, welcher automatisch eine vereinfachte Aerodynamik- und Strukturauslegung durchführt, und zwar wahlweise mittels eines Schalenflügel-Konzepts oder mittels Holm- und Rippenbauweise. Das Modul, welches die Leistungsberechnung anstellt, kann auch optimale Höhenprofile berücksichtigen, welche erlauben, potentielle Energie zu speichern. Die Ausgaben dieses Instrumentes zum Konzeptentwurf hat die Entwicklung des senseSoar Solarflugzeugs motiviert, welches mit weitreichender Sensorik und Bordrechnern ausgestattet worden ist. Wir beschreiben die Details des Entwicklungsprozesses welcher diverse Ingenieursdisziplinen überspannt, von Aerodynamik zu Solartechnologie, Elektronik bis hin zu Strukturauslegung. Des weiteren beschreiben wir eine modulare Sensor- und Recheneinheit, welche an einem weiteren Solarflugzeug angebracht werden kann, nämlich an AtlantikSolar. Im Entwicklungsprozess, aberauchfürdarauffolgendeSimulationensowiefürdieAuslegungvonAutopilotsystemen sind aerodynamische und flugmechanische Modelle von grosser Wichtigkeit; wir stellen eine komplette Werkzeugkette vor für entsprechende Analysen. Entwicklung effizienter Komponenten spielt eine Schlüsselrolle für den Erfolg eines Solarflugzeuges. Langzeiteinsätze können jedoch nur erreicht werden, wenn das Flugzeug zusätzlich genügend Robustheit aufweist. Diese beiden Konzepte haben ebenfalls Gültig- keit in bezug auf Algorithmen, welche es schliesslich zu einem selbständig agierenden System machen. Als Basis für jegliche Autonomie muss das Fluggerät sowohl seine inter- nen Zustandsvariablen, als auch seine Umgebung kennen, und zwar spezifisch in der Form einer Karte. Diese Dissertation behandelt in weiten Teilen genau entsprechende Heraus- forderungen, nämlich wie verschiedene Sensorsignale unter harten Echtzeitanforderungen und limitierter Rechenleistung verarbeitet werden können. Spezifisch werden zwei Algorithmen vorgestellt und im Detail analysiert, welche ein Element gemeinsam haben: nämlich Intertialmessungen, also gemessene Beschleunigungen vii