Studies in Systems, Decision and Control 17 Leonid Shaikhet Optimal Control of Stochastic Difference Volterra Equations An Introduction Studies in Systems, Decision and Control Volume 17 Series editor Janusz Kacprzyk, Polish Academy of Sciences, Warsaw, Poland e-mail: [email protected] About this Series The series “Studies in Systems, Decision and Control” (SSDC) covers both new developments and advances, as well as the state of the art, in the various areas of broadly perceived systems, decision making and control- quickly, up to date and withahighquality.Theintentistocoverthetheory,applications,andperspectives on the state of the art and future developments relevant to systems, decision making,control,complexprocessesandrelatedareas,asembeddedinthefieldsof engineering, computer science, physics, economics, social and life sciences, as well as the paradigms and methodologies behind them. The series contains monographs, textbooks, lecture notes and edited volumes in systems, decision making and control spanning the areas of Cyber-Physical Systems, Autonomous Systems, Sensor Networks, Control Systems, Energy Systems, Automotive Systems, Biological Systems, Vehicular Networking and Connected Vehicles, AerospaceSystems,Automation,Manufacturing,SmartGrids,NonlinearSystems, Power Systems, Robotics, Social Systems, Economic Systems and other. Of particular value to both the contributors and the readership are the short publicationtimeframeandtheworld-widedistributionandexposurewhichenable both a wide and rapid dissemination of research output. More information about this series at http://www.springer.com/series/13304 Leonid Shaikhet Optimal Control of Stochastic Difference Volterra Equations An Introduction 123 LeonidShaikhet Department of HigherMathematics Donetsk StateUniversity ofManagement Donetsk Ukraine ISSN 2198-4182 ISSN 2198-4190 (electronic) Studies inSystems,Decision and Control ISBN 978-3-319-13238-9 ISBN 978-3-319-13239-6 (eBook) DOI 10.1007/978-3-319-13239-6 LibraryofCongressControlNumber:2014955322 MathematicalSubjectCodeClassification(2010):37C75,93D05,93D20,93E15 SpringerChamHeidelbergNewYorkDordrechtLondon ©SpringerInternationalPublishingSwitzerland2015 Thisworkissubjecttocopyright.AllrightsarereservedbythePublisher,whetherthewholeorpart of the material is concerned, specifically the rights of translation, reprinting, reuse of illustrations, recitation, broadcasting, reproduction on microfilms or in any other physical way, and transmission or information storage and retrieval, electronic adaptation, computer software, or by similar or dissimilarmethodologynowknownorhereafterdeveloped. The use of general descriptive names, registered names, trademarks, service marks, etc. in this publicationdoesnotimply,evenintheabsenceofaspecificstatement,thatsuchnamesareexempt fromtherelevantprotectivelawsandregulationsandthereforefreeforgeneraluse. Thepublisher,theauthorsandtheeditorsaresafetoassumethattheadviceandinformationinthis book are believed to be true and accurate at the date of publication. Neither the publisher nor the authors or the editors give a warranty, express or implied, with respect to the material contained hereinorforanyerrorsoromissionsthatmayhavebeenmade. Printedonacid-freepaper Springer International Publishing AG Switzerland is part of Springer Science+Business Media (www.springer.com) Preface The aim of this book is an introduction to the mathematical theory of optimal control for a relatively new class of equations: stochastic difference Volterra equations of neutral type. Equations of such type arise as independent objects of research, as mathematical models of systems with discrete time, or as difference analogues of stochastic differential and integral Volterra equations of neutral type [9, 109] by their numerical simulation. These equations belong to the more general class of hereditary systems (also called systems with aftereffect, or systems with memory, or equations with devi- ating arguments, or equations with delays, or equations with time lag, and so on) and describe the processes whose behavior depends not only on their present state but also on their past history [9, 40, 42, 65–69, 72, 96, 103–106, 109, 138, 153, 154, 178, 179, 187, 199].Systemsof such type are very popular in researches and are widely used to model processes in physics, mechanics, automatic regulation, economy, finance, biology, ecology, sociology, medicine, etc. (see, e.g., [11, 15, 19,21–23,26,27,33,34,53,62,75,86,115,126,132,144,148,152,162,166– 169, 177–181, 188, 192, 197, 198]). The general theory of difference equations, including difference equations with delayisrepresentedin[2,3,19,25,38,39,56,59,95,98,121,123,135,151,156, 178,183].Alotofworksaredevoted,inparticular,toinvestigationofintegraland differenceVolterraequations[4,5,10,17,18,32,41,49–52,54,55,57,58,60,64, 76,81,93,94,101,102,108,117,118,133,143,145,157,189,190,193,213],to numerical analysis of systems with continuous time by virtue of appropriate dif- ference analogues [1, 35–39, 73, 83, 123, 129, 136, 137, 194, 195, 199], to description of different mathematical models of systems with discrete time [6, 7, 44–46, 78, 79, 82, 91, 92, 127, 128, 131, 139, 149, 150, 155, 206–209]. A very important and popular (in particular, in the last years) direction in the theory and different applications of systems of such type is the optimal control theory. To problems of existence and construction of optimal control in the sense ofthe given performance criterion andstabilization problems for deterministicand stochastic systems with continuous or discrete time, both without and with delays, v vi Preface plentyofworksaredevoted[8,9,12–14,16,20,23,24,28–30,32,43,47,48,61, 63, 68, 74, 77–80, 82, 84, 85, 87–91, 99, 109, 112–114, 116, 119, 122, 124, 125, 130,134,140–142,146,147,149,150,155,158–161,163–165,170–176,185,186, 191, 196, 200, 201, 203–212]. There are also many works devoted to problems of optimalestimation(see,forinstance,[17,18,31,97,100,107,111,120,182,202]). Inthisbook,consistingofsixchapters,alltheabove-mentionedbasicproblems of the mathematical theory of optimal control and optimal estimation are extended onstochasticdifferenceVolterraequationsofneutraltype.Itisshown,inparticular, that the difference analogues of the solutions of optimal control problems and optimal estimation problems obtained for stochastic integral Volterra equations cannot be optimal solutions of corresponding problems for stochastic difference Volterra equations. The introductory Chap. 1 presents an origin of stochastic difference Volterra equations of neutral type, a necessary condition for the optimality of a control for abstract optimal control problem and some auxiliary definitions and lemmas. In Chap. 2 a necessary condition for the control optimality of nonlinear sto- chastic difference Volterra equation is obtained and via this condition a synthesis of the optimal control for a linear-quadratic problem is constructed. A synthesis of the optimal control means the control constructed as a feedback control. Some demonstrative examples of calculating the optimal control in a final form are shown. In Chap. 3 the problem of construction of successive approximations to the optimal control of the stochastic quasilinear difference Volterra equations with quadraticperformancefunctionalisconsidered.Analgorithmforconstructingsuch approximations is described. It is shown that successive approximations can be considered both as a program control and as a feedback control. In Chap. 4 the problem of the optimal stabilization for a linear stochastic dif- ference Volterra equation and quadratic performance functional is considered. The optimal control in the sense of a given quadratic performance functional that sta- bilizes the solution of the considered equation to mean square stable and mean square summable is constructed. For the quasilinear stochastic difference Volterra equation with quadratic performance functional a zeroth approximation to the optimal control is constructed that stabilizes the solution of the considered differ- ence equation to mean square stable and mean square summable. In Chap. 5 the filtering problem is formulated. More exactly, the problem of constructingtheoptimal(inthemeansquaresense)estimateofanarbitrarypartially observable Gaussian stochastic process from its observations with delay is con- sidered.Itisprovedthatthedesiredestimateisdefinedbyauniquesolutionofthe fundamental filtering equation of the Wiener–Hopf type. Qualitative properties of thisequationarediscussedandseveralcaseswhereitcanbesolvedanalyticallyare considered. The relationship between the observation error and the magnitude of delay in observations is investigated. It is shown that the fundamental filtering equationdescribesalsothesolutionsoftheforecastingandinterpolationproblems. In the case where the unobservable process is given by a stochastic difference VolterraequationananalogueoftheKalman–Bucyfilterisconstructed:thesystem Preface vii offourstochasticdifferenceequationsdefinedtheoptimalinthemeansquaresense estimate. InChap.6twodifferentmethodsforsolutionoftheoptimalcontrolproblemfor partly observable linear stochasticprocesswithaquadraticperformancefunctional areproposed:theseparationmethodandthemethodofintegralrepresentations.An ε-optimal control of the optimal control problem for a quasilinear stochastic dif- ference Volterra equation and a quadratic performance functional is constructed. A special method is proposed for solution of the optimal control problem for stochastic linear difference equation with unknown parameter and quadratic per- formance functional. The optimal control in final form is obtained. Numerical calculations illustrate and continue the theoretical investigations. The bibliography at the end of the book does not pretend to be complete and includes some of the author’s publications and publications of his coauthors [31– 34,99,100,107–109,116–120,171–176,182]aswellastheliteratureusedbythe author during his preparation of this book. The bookismostly based ontheauthor’sresultsand ideas, closely relatedwith the results of other researchers in the theory of difference equations, the theory of hereditary systems, and the optimal control theory [2, 14, 19, 47, 55, 59, 68, 79, 103–106, 112, 130, 183]. It is addressed to experts in the mathematical optimal control theory as well as to a wider audience of professionals and students in pure and applied mathematics. Taking into account that the possibilities for further improvement and devel- opmentareendlesstheauthorwillappreciate receiving usefulremarks, comments, and suggestions. Donetsk, Ukraine Leonid Shaikhet Contents 1 Stochastic Difference Volterra Equations and Some Auxiliary Statements. . . . . . . . . . . . . . . . . . . . . . . . . . . 1 1.1 Origin of Stochastic Difference Volterra Equations of Neutral Type. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1 1.2 A Necessary Condition for Optimality of Control: Statement of the Problem . . . . . . . . . . . . . . . . . . . . . . . . . . . 2 1.3 Auxiliary Definitions and Assertions. . . . . . . . . . . . . . . . . . . . 4 1.4 Some Useful Lemmas. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10 2 Optimal Control. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13 2.1 A Necessary Condition for Optimality of Control for Nonlinear System . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13 2.1.1 Statement of the Problem . . . . . . . . . . . . . . . . . . . . . 13 2.1.2 Auxiliary Assertions. . . . . . . . . . . . . . . . . . . . . . . . . 16 2.1.3 Main Result. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32 2.2 A Linear-Quadratic Problem . . . . . . . . . . . . . . . . . . . . . . . . . 35 2.2.1 Synthesis of Optimal Control. . . . . . . . . . . . . . . . . . . 35 2.2.2 Examples . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 43 3 Successive Approximations to the Optimal Control . . . . . . . . . . . . 57 3.1 Statement of the Problem . . . . . . . . . . . . . . . . . . . . . . . . . . . 57 3.2 Algorithm of Successive Approximations Construction . . . . . . . 58 3.3 A Zeroth Approximation. . . . . . . . . . . . . . . . . . . . . . . . . . . . 68 3.4 Approximations of Higher Orders. . . . . . . . . . . . . . . . . . . . . . 71 4 Optimal and Quasioptimal Stabilization . . . . . . . . . . . . . . . . . . . . 79 4.1 Statement of the Linear Quadratic Optimal Stabilization Problem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 79 4.1.1 Auxiliary Stability Problem. . . . . . . . . . . . . . . . . . . . 81 4.1.2 Optimal Control Problem . . . . . . . . . . . . . . . . . . . . . 87 4.1.3 Stability by the Optimal Control. . . . . . . . . . . . . . . . . 103 ix x Contents 4.2 Quasioptimal Stabilization Problem. . . . . . . . . . . . . . . . . . . . . 111 4.2.1 Program Control . . . . . . . . . . . . . . . . . . . . . . . . . . . 113 4.2.2 Feedback Control. . . . . . . . . . . . . . . . . . . . . . . . . . . 119 5 Optimal Estimation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 127 5.1 Filtering Problem. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 127 5.1.1 Fundamental Filtering Equation . . . . . . . . . . . . . . . . . 127 5.1.2 Dual Problem of Optimal Control. . . . . . . . . . . . . . . . 133 5.1.3 Some Particular Cases. . . . . . . . . . . . . . . . . . . . . . . . 136 5.1.4 The Dependence of the Estimation Error on the Magnitude of Delay . . . . . . . . . . . . . . . . . . . . 138 5.2 Forecasting and Interpolation. . . . . . . . . . . . . . . . . . . . . . . . . 144 5.2.1 Statement and Solution of the Forecasting Problem . . . 144 5.2.2 Interpolation Problem. . . . . . . . . . . . . . . . . . . . . . . . 146 5.3 Filtration of Processes Described by Difference Equations. . . . . 147 5.3.1 An Analogue of the Kalman–Bucy Filter. . . . . . . . . . . 147 5.3.2 An Integral Representation of the Estimate . . . . . . . . . 154 5.4 Importance of Research of Difference Volterra Equations . . . . . 156 6 Optimal Control of Stochastic Difference Volterra Equations by Incomplete Information. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 159 6.1 Separation Method. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 159 6.1.1 Solution of the Filtering Problem. . . . . . . . . . . . . . . . 160 6.1.2 Solution of the Auxiliary Optimal Control Problem . . . 162 6.2 The Method of Integral Representations . . . . . . . . . . . . . . . . . 179 6.3 Quasilinear Stochastic Difference Volterra Equation . . . . . . . . . 187 6.4 Linear Stochastic Difference Equation with Unknown Parameter . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 191 6.4.1 Statement of the Problem . . . . . . . . . . . . . . . . . . . . . 191 6.4.2 The Optimal Control Construction . . . . . . . . . . . . . . . 193 6.4.3 The Optimal Cost Construction . . . . . . . . . . . . . . . . . 201 6.4.4 A Particular Case. . . . . . . . . . . . . . . . . . . . . . . . . . . 202 References. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 207 Index . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 217