IterativeLearningControlforMulti-agentSystemsCoordination Iterative Learning Control for Multi-agent Systems Coordination ShipingYang Jian-XinXu XuefangLi NationalUniversityofSingapore DongShen BeijingUniversityofChemicalTechnology,P.R.China Thiseditionfirstpublished2017 ©2017JohnWiley&SonsSingaporePte.Ltd RegisteredOffice JohnWiley&SonsSingaporePte.Ltd,1FusionopolisWalk,#07-01SolarisSouthTower,Singapore138628. Fordetailsofourglobaleditorialoffices,forcustomerservicesandforinformationabouthowtoapplyfor permissiontoreusethecopyrightmaterialinthisbookpleaseseeourwebsiteatwww.wiley.com. AllRightsReserved.Nopartofthispublicationmaybereproduced,storedinaretrievalsystemor transmitted,inanyformorbyanymeans,electronic,mechanical,photocopying,recording,scanning,or otherwise,exceptasexpresslypermittedbylaw,withouteitherthepriorwrittenpermissionofthePublisher, orauthorizationthroughpaymentoftheappropriatephotocopyfeetotheCopyrightClearanceCenter. RequestsforpermissionshouldbeaddressedtothePublisher,JohnWiley&SonsSingaporePte.Ltd., 1FusionopolisWalk,#07-01SolarisSouthTower,Singapore138628,tel:65-66438000,fax:65-66438008, email:[email protected]. Wileyalsopublishesitsbooksinavarietyofelectronicformats.Somecontentthatappearsinprintmaynot beavailableinelectronicbooks. Designationsusedbycompaniestodistinguishtheirproductsareoftenclaimedastrademarks.Allbrand namesandproductnamesusedinthisbookaretradenames,servicemarks,trademarksorregistered trademarksoftheirrespectiveowners.ThePublisherisnotassociatedwithanyproductorvendor mentionedinthisbook.Thispublicationisdesignedtoprovideaccurateandauthoritativeinformationin regardtothesubjectmattercovered.ItissoldontheunderstandingthatthePublisherisnotengagedin renderingprofessionalservices.Ifprofessionaladviceorotherexpertassistanceisrequired,theservicesofa competentprofessionalshouldbesought. LimitofLiability/DisclaimerofWarranty:Whilethepublisherandauthorhaveusedtheirbesteffortsin preparingthisbook,theymakenorepresentationsorwarrantieswithrespecttotheaccuracyor completenessofthecontentsofthisbookandspecificallydisclaimanyimpliedwarrantiesofmerchantability orfitnessforaparticularpurpose.Itissoldontheunderstandingthatthepublisherisnotengagedin renderingprofessionalservicesandneitherthepublishernortheauthorshallbeliablefordamagesarising herefrom.Ifprofessionaladviceorotherexpertassistanceisrequired,theservicesofacompetent professionalshouldbesought. LibraryofCongressCataloging-in-PublicationData Names:Yang,Shiping,1987–author.|Xu,Jian-Xin,author.|Li,Xuefang, 1985–author.|Shen,Dong,1982–author. Title:Iterativelearningcontrolformulti-agentsystemscoordination/by ShipingYang,Jian-XinXu,XuefangLi,DongShen. Description:Singapore:JohnWiley&Sons,Inc.,2017.|Includesbibliographicalreferencesandindex. Identifiers:LCCN2016052027(print)|LCCN2016056133(ebook)|ISBN9781119189046(hardback)| ISBN9781119189060(pdf)|ISBN9781119189077(epub) Subjects:LCSH:Intelligentcontrolsystems.|Multiagentsystems.|Machinelearning.|Iterativemethods (Mathematics)|BISAC:TECHNOLOGY&ENGINEERING/Robotics. Classification:LCCTJ217.5.Y362017(print)|LCCTJ217.5(ebook)|DDC629.8/9–dc23 LCrecordavailableathttps://lccn.loc.gov/2016052027 AcatalogrecordforthisbookisavailablefromtheBritishLibrary. Setin10/12ptWarnockbySPiGlobal,Pondicherry,India Coverdesign:Wiley Coverimage:©MichaelMann/Gettyimages 10 9 8 7 6 5 4 3 2 1 v Contents Preface ix 1 Introduction 1 1.1 IntroductiontoIterativeLearningControl 1 1.1.1 Contraction-MappingApproach 3 1.1.2 CompositeEnergyFunctionApproach 4 1.2 IntroductiontoMASCoordination 5 1.3 MotivationandOverview 7 1.4 CommonNotationsinThisBook 9 2 OptimalIterativeLearningControlforMulti-agentConsensus Tracking 11 2.1 Introduction 11 2.2 PreliminariesandProblemDescription 12 2.2.1 Preliminaries 12 2.2.2 ProblemDescription 13 2.3 MainResults 15 2.3.1 ControllerDesignforHomogeneousAgents 15 2.3.2 ControllerDesignforHeterogeneousAgents 20 2.4 OptimalLearningGainDesign 21 2.5 IllustrativeExample 23 2.6 Conclusion 26 3 IterativeLearningControlforMulti-agentCoordinationUnder Iteration-VaryingGraph 27 3.1 Introduction 27 3.2 ProblemDescription 28 3.3 MainResults 29 3.3.1 FixedStronglyConnectedGraph 29 3.3.2 Iteration-VaryingStronglyConnectedGraph 32 3.3.3 UniformlyStronglyConnectedGraph 37 3.4 IllustrativeExample 38 3.5 Conclusion 40 vi Contents 4 IterativeLearningControlforMulti-agentCoordinationwithInitial StateError 41 4.1 Introduction 41 4.2 ProblemDescription 42 4.3 MainResults 43 4.3.1 DistributedD-typeUpdatingRule 43 4.3.2 DistributedPD-typeUpdatingRule 48 4.4 IllustrativeExamples 49 4.5 Conclusion 50 5 Multi-agentConsensusTrackingwithInputSharingbyIterative LearningControl 53 5.1 Introduction 53 5.2 ProblemFormulation 54 5.3 ControllerDesignandConvergenceAnalysis 54 5.3.1 ControllerDesignWithoutLeader’sInputSharing 55 5.3.2 OptimalDesignWithoutLeader’sInputSharing 58 5.3.3 ControllerDesignwithLeader’sInputSharing 59 5.4 ExtensiontoIteration-VaryingGraph 60 5.4.1 Iteration-VaryingGraphwithSpanningTrees 60 5.4.2 Iteration-VaryingStronglyConnectedGraph 60 5.4.3 UniformlyStronglyConnectedGraph 62 5.5 IllustrativeExamples 63 5.5.1 Example1:Iteration-InvariantCommunicationGraph 63 5.5.2 Example2:Iteration-VaryingCommunicationGraph 64 5.5.3 Example3:UniformlyStronglyConnectedGraph 66 5.6 Conclusion 68 6 AHOIM-BasedIterativeLearningControlSchemeforMulti-agent Formation 69 6.1 Introduction 69 6.2 KinematicModelFormulation 70 6.3 HOIM-BasedILCforMulti-agentFormation 71 6.3.1 ControlLawforAgent1 72 6.3.2 ControlLawforAgent2 74 6.3.3 ControlLawforAgent3 75 6.3.4 SwitchingBetweenTwoStructures 78 6.4 IllustrativeExample 78 6.5 Conclusion 80 7 P-typeIterativeLearningforNon-parameterizedSystemswith UncertainLocalLipschitzTerms 81 7.1 Introduction 81 7.2 MotivationandProblemDescription 82 7.2.1 Motivation 82 7.2.2 ProblemDescription 83 7.3 ConvergencePropertieswithLyapunovStabilityConditions 84 Contents vii 7.3.1 PreliminaryResults 84 7.3.2 LyapunovStableSystems 86 7.3.3 SystemswithStableLocalLipschitzTermsbutUnstableGlobal LipschitzFactors 90 7.4 ConvergencePropertiesinthePresenceofBoundingConditions 92 7.4.1 SystemswithBoundedDriftTerm 92 7.4.2 SystemswithBoundedControlInput 94 7.5 ApplicationofP-typeRuleinMASwithLocalLipschitzUncertainties 97 7.6 Conclusion 99 8 SynchronizationforNonlinearMulti-agentSystemsbyAdaptive IterativeLearningControl 101 8.1 Introduction 101 8.2 PreliminariesandProblemDescription 102 8.2.1 Preliminaries 102 8.2.2 ProblemDescriptionforFirst-OrderSystems 102 8.3 ControllerDesignforFirst-OrderMulti-agentSystems 105 8.3.1 MainResults 105 8.3.2 ExtensiontoAlignmentCondition 107 8.4 ExtensiontoHigh-OrderSystems 108 8.5 IllustrativeExample 113 8.5.1 First-OrderAgents 114 8.5.2 High-OrderAgents 115 8.6 Conclusion 118 9 DistributedAdaptiveIterativeLearningControlforNonlinear Multi-agentSystemswithStateConstraints 123 9.1 Introduction 123 9.2 ProblemFormulation 124 9.3 MainResults 127 9.3.1 OriginalAlgorithms 127 9.3.2 ProjectionBasedAlgorithms 135 9.3.3 SmoothFunctionBasedAlgorithms 138 9.3.4 AlternativeSmoothFunctionBasedAlgorithms 141 9.3.5 PracticalDead-ZoneBasedAlgorithms 145 9.4 IllustrativeExample 147 9.5 Conclusion 174 10 SynchronizationforNetworkedLagrangianSystemsunder DirectedGraphs 175 10.1 Introduction 175 10.2 ProblemDescription 176 10.3 ControllerDesignandPerformanceAnalysis 177 10.4 ExtensiontoAlignmentCondition 183 10.5 IllustrativeExample 184 10.6 Conclusion 188 viii Contents 11 GeneralizedIterativeLearningforEconomicDispatchProblemina SmartGrid 189 11.1 Introduction 189 11.2 Preliminaries 190 11.2.1 In-NeighborandOut-Neighbor 190 11.2.2 Discrete-TimeConsensusAlgorithm 191 11.2.3 AnalyticSolutiontoEDPwithLossCalculation 192 11.3 MainResults 193 11.3.1 UpperLevel:EstimatingthePowerLoss 194 11.3.2 LowerLevel:SolvingEconomicDispatchDistributively 194 11.3.3 GeneralizationtotheConstrainedCase 197 11.4 LearningGainDesign 198 11.5 ApplicationExamples 200 11.5.1 CaseStudy1:ConvergenceTest 201 11.5.2 CaseStudy2:RobustnessofCommandNodeConnections 202 11.5.3 CaseStudy3:PlugandPlayTest 203 11.5.4 CaseStudy4:Time-VaryingDemand 205 11.5.5 CaseStudy5:ApplicationinLargeNetworks 207 11.5.6 CaseStudy6:RelationBetweenConvergenceSpeedandLearningGain 207 11.6 Conclusion 208 12 SummaryandFutureResearchDirections 209 12.1 Summary 209 12.2 FutureResearchDirections 210 12.2.1 OpenIssuesinMASControl 210 12.2.2 Applications 214 AppendixAGraphTheoryRevisit 223 AppendixBDetailedProofs 225 B.1 HOIMConstraintsDerivation 225 B.2 ProofofProposition2.1 226 B.3 ProofofLemma2.1 227 B.4 ProofofTheorem8.1 229 B.5 ProofofCorollary8.1 230 Bibliography 233 Index 245 ix Preface Thecoordinationandcontrolproblemsofmulti-agentsystems(MAS)havebeenexten- sively studied by the control community due to the broad practical applications, for example,theformationcontrolproblem,searchandrescuebymultipleaerialvehicles, synchronization,sensorfusion,distributedoptimization,theeconomicdispatchprob- lem in power systems, and so on. Meanwhile, many industry processes require both repetitiveexecutionsandcoordinationamongseveralindependententities.Thisobser- vation motivates the research of multi-agent coordination from an iterative learning control(ILC)perspective.Thisbookisdedicatedtotheapplicationofiterativelearning controltomulti-agentcoordinationproblems. Inordertostudymulti-agentcoordinationbyILC,anextradimension,theiteration domain,isintroducedintotheproblem.Achallengingissueincontrollingmulti-agent systems by ILC is the non-perfect repeating characteristics of MAS. The inherent nature of MAS such as heterogeneity, information sharing, sparse and intermittent communication, imperfect initial conditions, and inconsistent target tracking trajec- toriesincreasesthecomplexityoftheproblem.Duetothesefactors,controllerdesign becomesachallengingproblem.Thisbookprovidesdetailedguidelinesforthedesignof learningcontrollersundervariouscoordinationconditions,inasystematicmanner.The maincontentcanbeclassifiedintotwopartsfollowingthetwomainframeworksofILC, namely,contraction-mapping(CM)andcompositeenergyfunction(CEF)approaches. Chapters 2–7 apply the CM approach, while Chapters 8–10 apply the CEF approach. Eachchapterstudiesthecoordinationproblemundercertainconditions.Forexample, Chapter 2 assumes a fixed communication topology, Chapter 3 assumes a switching topology, and Chapter 4 addresses the initial state error problem in multi-agent coordination.Inasense,eachchapteraddressesauniquecoordinationcontrolproblem forMAS.Chapters2–10discusscontinuous-timesystems.InChapter11wepresenta generalizediterativelearningalgorithmtosolveanoptimalpowerdispatchproblemin asmartgridbyutilizingdiscrete-timeconsensusalgorithms. This book is self contained and intensive. Prior knowledge of ILC and MAS is not required.Chapter1providesarudimentaryintroductiontothetwoareas.Twominimal examplesofILCarepresentedinChapter1,andashortreviewofsometerminologies in graph theory is provided in AppendixA. Readers can skip the preliminary parts if they are familiar with the domain. We present detailed convergence proofs for each controlleraswebelievethatunderstandingthetheoreticalderivationscanbenefitread- ers in two ways. On the one hand, it helps readers appreciate the controller design. On the other hand, the control design and analysis techniques can be transferred to x Preface otherdomainstofacilitatefurtherexplorationinvariouscontrolapplications.Specifi- callyforindustrialexpertsandpractitioners,weprovidedetailedillustrativeexamples ineachchaptertoshowhowthosecontrolalgorithmsareimplemented.Theexamples demonstratetheeffectivenessofthelearningcontrollersandcanbemodifiedtohandle practicalproblems. 1 1 Introduction 1.1 IntroductiontoIterativeLearningControl Iterativelearningcontrol(ILC),asaneffectivecontrolstrategy,isdesignedtoimprove current control performance for unpredictable systems by fully utilizing past control experience. Specifically, ILC is designed for systems that complete tasks over a fixed time interval and perform them repeatedly. The underlying philosophy mimics the human learning process that “practice makes perfect.” By synthesizing control inputs from previous control inputs and tracking errors, the controller is able to learn from pastexperienceandimprovecurrenttrackingperformance.ILCwasinitiallydeveloped byArimotoetal.(1984),andhasbeenwidelyexploredbythecontrolcommunitysince then(Moore,1993;BienandXu,1998;ChenandWen,1999;Longman,2000;Norrlof andGunnarsson,2002;XuandTan,2003;Bristowetal.,2006;Mooreetal.,2006;Ahn etal., 2007a; Rogers etal., 2007; Ahn etal., 2007b; Xu etal., 2008; Wang etal., 2009, 2014). Figure1.1 shows the schematic diagram of an ILC system, where the subscript i denotestheiterationindexandy denotesthereferencetrajectory.Basedontheinput d signal, u, at the ith iteration, as well as the tracking error e =y −y, the input u i i d i i+1 for the next iteration, namely the (i+1)th iteration, is constructed. Meanwhile, the inputsignalu willalsobestoredintomemoryforuseinthe(i+2)thiteration.Itis i+1 importanttonotethatinFigure1.1,aclosedfeedbackloopisformedintheiteration domain rather than the time domain. Compared to other control methods such as proportional-integral-derivative (PID) control and sliding mode control, there are a number of distinctive features about ILC. First, ILC is designed to handle repetitive control tasks, while other control techniques don’t typically take advantage of task repetition—under a repeatable control environment, repeating the same feedback wouldyieldthesamecontrolperformance.Incontrast,byincorporatinglearning,ILC isabletoimprovethecontrolperformanceiteratively.Second,thecontrolobjectiveis different. ILC aims at achieving perfect tracking over the whole operational interval. Most control methods aim to achieve asymptotic convergence in tracking accuracy over time. Third, ILC is a feedforward control method if viewed in the time domain. The plant shown in Figure1.1 is a generalized plant, that is, it can actually include a feedbackloop.ILCcanbeusedtofurtherimprovetheperformanceofthegeneralized plant.Assuch,thegeneralizedplantcouldbemadestableinthetimedomain,which is helpful in guaranteeing transient response while learning takes place. Last but not IterativeLearningControlforMulti-agentSystemsCoordination,FirstEdition. ShipingYang,Jian-XinXu,XuefangLi,andDongShen. ©2017JohnWiley&SonsSingaporePte.Ltd.Published2017byJohnWiley&SonsSingaporePte.Ltd.