UNIVERSITY OF OSLO Department of Informatics Trajectory Learning for Highly Aerobatic Unmanned Aerial Vehicle Master Thesis Maja Celine Sevaldson August 14, 2012 Abstract Thedynamicsoffixedwingplanesarewellunderstood withinthe conven- tional flight envelope. The situation is different in the case of aerobatic maneuverswithalargeangleofattack,suchasperchingandverticalhover. In such maneuvers the airflows around the plane are unpredictable mak- ingitdifficulttocreateaccuratedynamicmodels,whichwouldnormallybe neededforthedesignofconventionalcontrollers. YethumanRCpilotsare abletoflythesemaneuverswithfixedwingplanes. Apprenticeship learning provides a promising solution to the problem of automating highly aerobatic maneuvers. It allows the maneuver to be learnedfrom demonstration flightsdone by a human RC pilotrather than relyingonanaccuratedynamicsmodel. Thefocusofthisthesisisonaspecificissueinapprenticeshiplearning, namely how to infer the trajectory the pilot intended to fly from a set of suboptimal demonstration trajectories. Such a trajectory can be used as a target trajectory for an autonomous controller. A trajectory learning algorithm that has shown promising results in automation of aerobatic helicopterflightisappliedtoafixedwingUAVplatform. The algorithm is tested on two different maneuvers; A straight line of level flight, and a vertical hover maneuver. In the case of both maneuvers the algorithm learned the intended trajectory without prior knowledge aboutthetrajectory. In order to collect training data for the trajectory learning task, an appropriate platform and data acquisition system are needed. This thesis therefore also presents the development of a fixed wing UAV platform for researchonautomationofaerobaticflight. iii iv Contents 1 Introduction 1 1.1 Motivation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1 1.2 Thesisoverview . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1 1.2.1 Supportingcontributions . . . . . . . . . . . . . . . . . . 2 1.2.2 Thesisoutline . . . . . . . . . . . . . . . . . . . . . . . . . 2 1.3 Relatedwork . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3 1.3.1 StanfordAutonomousHelicopter . . . . . . . . . . . . . 3 1.3.2 Aerobaticmaneuvers . . . . . . . . . . . . . . . . . . . . 4 2 Background 7 2.1 HiddenMarkovModels . . . . . . . . . . . . . . . . . . . . . . . 7 2.1.1 DiscreteMarkovProcesses . . . . . . . . . . . . . . . . . 7 2.1.2 HiddenMarkovModel. . . . . . . . . . . . . . . . . . . . 8 2.2 Expectation-Maximization. . . . . . . . . . . . . . . . . . . . . . 10 2.2.1 MaximumLikelihoodEstimation . . . . . . . . . . . . . 11 2.2.2 EM . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11 2.3 ExtendedKalmanSmoother. . . . . . . . . . . . . . . . . . . . . 12 2.3.1 DiscreteKalmanfilter . . . . . . . . . . . . . . . . . . . . 12 2.3.2 ExtendedKalmanfilter . . . . . . . . . . . . . . . . . . . 13 2.3.3 Kalmansmoother . . . . . . . . . . . . . . . . . . . . . . 13 2.4 DynamicTimeWarping . . . . . . . . . . . . . . . . . . . . . . . 14 3 TrajectoryLearning 17 3.1 Statemodelling . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17 3.1.1 Timealignment . . . . . . . . . . . . . . . . . . . . . . . . 18 3.2 Thetrajectorylearningalgorithm . . . . . . . . . . . . . . . . . 19 3.2.1 E-stepforhiddentrajectory . . . . . . . . . . . . . . . . 20 3.2.2 M-stepforhiddentrajectory . . . . . . . . . . . . . . . . 20 3.2.3 Timealignment . . . . . . . . . . . . . . . . . . . . . . . . 21 3.3 DynamicModel . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22 3.3.1 Equationsofmotion . . . . . . . . . . . . . . . . . . . . . 22 3.4 Improvementsofthemodel . . . . . . . . . . . . . . . . . . . . . 23 4 Experimentalplatform 25 4.1 Plane . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26 4.1.1 Dynamicalpropertiesoftheplane. . . . . . . . . . . . . 27 4.1.2 Modificationstotheairframe . . . . . . . . . . . . . . . 29 v 4.1.3 Motor . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29 4.1.4 RadioControlSystem . . . . . . . . . . . . . . . . . . . . 30 4.2 Microcontrollers,SensorsandDataAcquisition . . . . . . . . 31 4.2.1 ArduPilotMega . . . . . . . . . . . . . . . . . . . . . . . . 32 4.2.2 InertialMeasurementUnit . . . . . . . . . . . . . . . . . 32 4.2.3 GPS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33 4.2.4 Barometricpressuresensor. . . . . . . . . . . . . . . . . 34 4.2.5 Sonar . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 36 4.2.6 PilotInputs . . . . . . . . . . . . . . . . . . . . . . . . . . 36 4.2.7 DataAcquisitionandStorage . . . . . . . . . . . . . . . 37 4.3 Testenvironment . . . . . . . . . . . . . . . . . . . . . . . . . . . 37 4.4 AlternativePositioningSystems . . . . . . . . . . . . . . . . . . 39 4.5 Simulator . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41 5 ExperimentsandResults 43 5.1 Preliminarytests. . . . . . . . . . . . . . . . . . . . . . . . . . . . 43 5.1.1 GPSaccuracy . . . . . . . . . . . . . . . . . . . . . . . . . 43 5.1.2 Outdoorflighttests . . . . . . . . . . . . . . . . . . . . . . 44 5.2 Collectingtrainingdata. . . . . . . . . . . . . . . . . . . . . . . . 45 5.2.1 Linearflightandbankedturns . . . . . . . . . . . . . . . 46 5.2.2 Hoveringwithtransitions . . . . . . . . . . . . . . . . . . 47 5.3 Pre-processing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 47 5.3.1 Selectinggooddemonstrations . . . . . . . . . . . . . . 49 5.3.2 BPMfilteringandcalibration. . . . . . . . . . . . . . . . 49 5.3.3 LocalizationofPositionData . . . . . . . . . . . . . . . . 51 5.3.4 IMUdata . . . . . . . . . . . . . . . . . . . . . . . . . . . . 52 5.3.5 Pilotcontrolinputs . . . . . . . . . . . . . . . . . . . . . . 55 5.4 Results. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 57 5.4.1 Timealignment . . . . . . . . . . . . . . . . . . . . . . . . 57 5.4.2 Straightlineoflevelflight . . . . . . . . . . . . . . . . . . 59 5.4.3 Verticalhover . . . . . . . . . . . . . . . . . . . . . . . . . 62 5.4.4 ModelPredictionError . . . . . . . . . . . . . . . . . . . 64 6 Conclusion 67 6.1 Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 67 6.1.1 Platform . . . . . . . . . . . . . . . . . . . . . . . . . . . . 67 6.1.2 Timealignment . . . . . . . . . . . . . . . . . . . . . . . . 68 6.1.3 Trajectorylearning . . . . . . . . . . . . . . . . . . . . . . 68 6.2 Futurework . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 69 6.3 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 70 vi List of Figures 2.1 ThreestateMarkovmodeloftheweather . . . . . . . . . . . . . 9 2.2 HiddenMarkovModelofcrazysodadispenser . . . . . . . . . 9 2.3 Dynamictimewarping. Thefigureisadaptedfrom[31]. . . . . 14 2.4 Dynamic time warping distance matrix. The figure is borrowedfrom[40]. . . . . . . . . . . . . . . . . . . . . . . . . . 15 3.1 Modeloftherelationbetweenthehiddentrajectoryandone ofthedemonstrations. Thecolourednodesareunobserved. . 19 3.2 Model of the state when τk is fixed. Coloured nodes are j unobserved.. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19 4.1 Expertpilotflyingtheplaneinverticalhover. . . . . . . . . . . 25 4.2 ElectriflyYak543D . . . . . . . . . . . . . . . . . . . . . . . . . . 26 4.3 Reinforcementofthemotormount . . . . . . . . . . . . . . . . 30 4.4 Measuredthrustasafunctionofcontrolinput. . . . . . . . . . 30 4.5 7channelFutabaRCsystem . . . . . . . . . . . . . . . . . . . . . 31 4.6 DiagramofDAQsystem . . . . . . . . . . . . . . . . . . . . . . . 32 4.7 Electronicsmountedontheplane . . . . . . . . . . . . . . . . . 33 4.8 ArduPilotMegaboard . . . . . . . . . . . . . . . . . . . . . . . . 34 4.9 MicroStrain3DM-GX3IMU . . . . . . . . . . . . . . . . . . . . . 34 4.10 MediaTek3329GPS . . . . . . . . . . . . . . . . . . . . . . . . . 35 4.11 Bosch Sensortech BMP085 digital pressure sensor inside plasticenclosure . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35 4.12 Maxbotixultrasonicrangefindersonar . . . . . . . . . . . . . . 36 4.13 Outdoortestspace. . . . . . . . . . . . . . . . . . . . . . . . . . . 37 4.14 IndoorflightintheCEPSUMgym. . . . . . . . . . . . . . . . . . 38 4.15 TheStingerDomeofConcordiaUniversityinMontreal . . . . 38 4.16 Principleofstereocomputervision . . . . . . . . . . . . . . . . 39 4.17 2Dtrackingbasedoncolourtresholding . . . . . . . . . . . . . 40 5.1 On ground GPS test at McGill downtown campus. The red line marks the actual path travelled, while the yellow line showstheloggedGPSdata. . . . . . . . . . . . . . . . . . . . . . 43 5.2 ComparisonofGPSandBPMaltitudedata . . . . . . . . . . . . 44 5.3 InairtestsofDAQsystemandGPSlogging. . . . . . . . . . . . 45 5.4 Handlaunchoftheplaneforatrainingdatacollectionflight intheStingerDome. . . . . . . . . . . . . . . . . . . . . . . . . . 46 5.5 Mark-upoftheStingerDomefieldforcollectingtrainingdata. 47 vii 5.6 VerticalhoverintheStingerDome. . . . . . . . . . . . . . . . . 48 5.7 Verticalhover. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 48 5.8 East-Northplotofagoodlinearflightdemonstration. . . . . 49 5.9 Positionplotandpitchanglefromagoodhoverdemonstration 50 5.10 BPMdataisfilteredusinga3rdorderButterworth low-pass filterwith1Hzcut-offfrequency. . . . . . . . . . . . . . . . . . . 50 5.11 UncalibratedBPMdataandSonarreferencesignal . . . . . . . 51 5.12 Uncalibrated BPM data and Sonar reference at the two regionswherecalibrationpointswereselected. . . . . . . . . . 51 5.13 CalibratedBPMdataandSonarreferencesignal . . . . . . . . 52 5.14 Zoomedinviewofthetwocalibrationregionsaftercalibration 52 5.15 IMUframeandbodyframeoftheplane. . . . . . . . . . . . . . 53 5.16 Ailerondeflectionasafunctionofinputpulsewidth . . . . . . 55 5.17 Elevatordeflectionasafunctionofinputpulsewidth . . . . . 56 5.18 Rudderdeflectionasafunctionofinputpulsewidth . . . . . . 56 5.19 ThrustinNewtonsvsPWMcommand . . . . . . . . . . . . . . 57 5.20 Linearvelocities(n˙e˙d˙)fromthelinearflight. . . . . . . . . . . 58 5.21 AngularVelocitiesaroundtheplanez-axis(r). . . . . . . . . . . 58 5.22 East-Northpositionoflevelflight(meters) . . . . . . . . . . . . 59 5.23 Positionandlinearvelocityoftheplaneduringastraightline oflevelflight. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 60 5.24 Orientationduringlevelflight(EulerAngles). . . . . . . . . . . 60 5.25 Angularvelocitiespqr (rad/sec),levelflight.. . . . . . . . . . . 61 5.26 Controlinputs,levelflight. . . . . . . . . . . . . . . . . . . . . . 61 5.27 Hovermaneuver. . . . . . . . . . . . . . . . . . . . . . . . . . . . 62 5.28 Hovermaneuver-Linearvelocitiesandposition. . . . . . . . . 63 5.29 AngularVelocities,hoverflight(rad/sec). . . . . . . . . . . . . 63 5.30 Orientationduringhover(Eulerangles). . . . . . . . . . . . . . 63 5.31 Controlinputs,hovermaneuver. . . . . . . . . . . . . . . . . . . 64 5.32 Model prediction error inlinearaccelerations (u˙v˙w˙) during straightflight. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 65 5.33 Modelpredictionerrorinangularaccelerations(p˙q˙r˙)during straightflight. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 65 5.34 Model prediction error inlinearaccelerations (u˙v˙w˙) during thehovermaneuver. . . . . . . . . . . . . . . . . . . . . . . . . . 65 5.35 Modelpredictionerrorinangularaccelerations(p˙q˙r˙)during thehovermaneuver. . . . . . . . . . . . . . . . . . . . . . . . . . 65 viii List of Tables 4.1 Detailedweightbreakdownoftheexperimentalplatform . . . 27 4.2 MomentsandproductsofinertiaofYak54withData Acqui- sitionsystem. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28 4.3 Parameters used in calculation of rotational damping mo- ments. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29 ix x