Applied Probability Applications of Control Mathematics Economics Iriformation and Communication Modeling and Identification 20 Numerical Techniques Optimization Edited by A. V. Balakrishnan Applications of Mathematics I Fleming/Rishel, Deterministic and Stochastic Optimal Control (1975) 2 Marchuk, Methods of Numerical Mathematics, Second Ed. (1982) 3 Balakrishnan, Applied Functional Analysis, Second Ed. (1981) 4 Borovkov, Stochastic Processes in Queueing Theory (1976) 5 Lipster/Shiryayev, Statistics of Random Processes I: General Theory (1977) 6 Lipster/Shiryayev, Statistics of Random Processes II: Applications (1978) 7 Vorob' ev, Game Theory: Lectures for Economists and Systems Scientists (1977) 8 Shiryayev, Optimal Stopping Rules (1978) 9 Ibragimov/Rozanov, Gaussian Random Processes (1978) 10 Wonham, Linear Multivariable Control: A Geometric Approach, Third Ed. (1985) II Hida, Brownian Motion (1980) 12 Hestenes, Conjugate Direction Methods in Optimization (1980) 13 Kallianpur, Stochastic Filtering Theory (1980) 14 Krylov, Controlled Diffusion Processes (1980) 15 Prabhu, Stochastic Storage Processes: Queues, Insurance Risk, and Dams (1980) 16 Ibragimov/Has'minskii, Statistical Estimation: Asymptotic Theory (1981) 17 Cesari, Optimization: Theory and Applications (1982) 18 Elliott, Stochastic Calculus and Applications (1982) 19 MarchukiShaidourov, Difference Methods and Their Extrapolations (1983) 20 Hijab, Stabilization of Control Systems (1986) o. Hijab Stabilization of Control Systems Springer Science+Business Media, LLC O. Hijab Mathematics Department Temple University Philadelphia, PA 19122 U.S.A. Managing Editor A. V. Balakrishnan Systems Science Department University of California Los Angeles, CA 90024 U.S.A. AMS Classification: 93EXX With 3 Illustrations Library of Congress Cataloging in Publication Data Hijab, O. Stabilization of control systems. (Applications of mathematics; 20) Includes bibliographical references and index. 1. System analysis. 2. Stochastic systems. 3. Stability. I. Title. II. Series. QA402.H55 1986 003 86-13920 © Springer Science+Business Media New York 1987 Originally published by Springer-Verlag New York, Inc. in 1987 Softcover reprint of the hardcover 1st edition 1987 All rights reserved. No part of this book may be translated or reproduce d in any form without written permission from Springer Science+Business Media, LLC. Typeset by Asco Trade Typesetting Ltd., Hong Kong. 9 8 7 6 5 4 3 21 ISBN 978-1-4419-3080-4 ISBN 978-1-4899-0013-5 (eBook) DOI 10.1007/978-1-4899-0013-5 To Carol Armstrong, who makes it all worthwhile u x z (Ae, Be) Ce + noise u y feedback controller given below ..... Xl ) I u y F .. Feedback Control Appearing in Chapter 5 Contents Introduction ix Notation xi CHAPTER 1 Input/Output Properties 1.1. An Example 1 1.2. Review of Linear Algebra 4 1.3. Linear Systems 8 1.4. Controllability and Observability 10 1.5. Minimality 13 1.6. Realizability 16 1.7. Notes and References 19 CHAPTER 2 The LQ Regulator 21 2.1. Stabilization 21 2.2. Properness 24 2.3. Optimal Control 28 2.4. The Riccati Equation 32 2.5. The Space M(m, n, p) 35 2.6. Notes and References 41 CHAPTER 3 Brownian Motion 43 3.1. Preliminary Definitions 43 3.2. Stochastic Calculus 52 3.3. Cameron-Martin-Girsanov Formula 57 3.4. Notes and References 62 viii Contents CHAPTER 4 Filtering 64 4.1. Filtering 64 4.2. Consistency 73 4.3. Shannon Information 77 4.4. Notes and References 82 CHAPTERS The Adaptive LQ Regulator 84 5.1. Introduction 84 5.2. Smooth Admissible Controls 87 5.3. Adaptive Stabilization 91 5.4. Optimal Control 94 5.5. Bellman Equation 98 5.6. Notes and References 102 APPENDIX Solutions to Exercises 103 Index 127 Introduction The problem of controlling or stabilizing a system of differential equa tions in the presence of random disturbances is intuitively appealing and has been a motivating force behind a wide variety of results grouped loosely together under the heading of "Stochastic Control." This book is concerned with a special instance of this general problem, the "Adaptive LQ Regulator," which is a stochastic control problem of partially observed type that can, in certain cases, be solved explicitly. We first describe this problem, as it is the focal point for the entire book, and then describe the contents of the book. The problem revolves around an uncertain linear system x(O) = x~ in R", where 0 E {1, ... , N} is a random variable representing this uncertainty and xJ (Ai' Bj, Cj) and are the coefficient matrices and initial state, respectively, of a linear control system, for eachj = 1, ... , N. A common assumption is that the mechanism causing this uncertainty is additive noise, and that conse quently the "controller" has access only to the observation process y( . ) where y = Cex +~. Here ~(.) is the proverbial "white noise," the fictional (time) derivative of Brownian motion. The problem then is to seek a causal feedback u(t) = F(t, y(s), 0 ~ s ~ t), t ~ 0, that minimizes a cost functional of the type J(u) = E(L'Xl1U(tW + Ix(tW dt) + H(oo)