Graduate Texts in Mathematics 45 Editorial Board F. W. Gehring P.R. Halmos Managing Editor C. C. Moore M. Loeve Probability Theory I 4th Edition Springer Science+Business Media, LLC M. Loeve Departments of Mathematics and Statistics University of California at Berkeley Berkeley, California 94720 Editorial Board P.R. Halmos F. W. Gehring C. C. Moore Managing Editor University of Michigan University of California at Berkeley University of California Department of Mathematics Department of Mathematics Department of Mathematics Ann Arbor, Michigan 48104 Berkeley, California 94720 Santa Barbara, California 93106 AMS Subject Classifications 28-01, 60A05, 60Bxx, 60E05, 60Fxx Library of Congress Cataloging in Publication Data Loeve, Michel, 1907- Probability theory. (Graduate texts in mathematics; 45) Bibliography p. Includes index. 1. Probabilities. I. Title. II. Series. QA273.L63 1977 519.2 76-28332 All rights reserved. No part of this book may be translated or reproduced in any form without written permission from Springer-Verlag. © 1963 by M. Loeve © 1977 by Springer Science+Business Media New York Originally published by Springer-Verlag New York in 1977 Origipally published in the University Series in Higher Mathematics (D. Van Nostrand Company); edited by M. H. Stone, L. Nirenberg, and S. S. Chern. ISBN 978-0-387-90127-5 ISBN 978-1-4757-6288-4 (eBook) DOl 10.1007/978-1-4757-6288-4 To LINE and To the students and teachers of the School in the Camp de Draney PREFACE TO THE FOURTH EDITION This fourth edition contains several additions. The main ones con cern three closely related topics: Brownian motion, functional limit distributions, and random walks. Besides the power and ingenuity of their methods and the depth and beauty of their results, their importance is fast growing in Analysis as well as in theoretical and applied Proba bility. These additions increased the book to an unwieldy size and it had to be split into two volumes. About half of the first volume is devoted to an elementary introduc tion, then to mathematical foundations and basic probability concepts and tools. The second half is devoted to a detailed study of Independ ence which played and continues to play a central role both by itself and as a catalyst. The main additions consist of a section on convergence of probabilities on metric spaces and a chapter whose first section on domains of attrac tion completes the study of the Central limit problem, while the second one is devoted to random walks. About a third of the second volume is devoted to conditioning and properties of sequences of various types of dependence. The other two thirds are devoted to random functions; the last Part on Elements of random analysis is more sophisticated. The main addition consists of a chapter on Brownian motion and limit distributions. It is strongly recommended that the reader begin with less involved portions. In particular, the starred ones ought to be left out until they are needed or unless the reader is especially interested in them. I take this opportunity to thank Mrs. Rubalcava for her beautiful typing of all the editions since the inception of the book. I also wish to thank the editors of Springer-Verlag, New York, for their patience and care. M.L. January, 1977 Berkeley, California PREFACE TO THE THIRD EDITION This book is intended as a text for graduate students and as a reference for· workers in Probability and Statistics. The prerequisite is honest calculus. The material covered in Parts Two to Five inclusive requires about three to four semesters of graduate study. The introductory part may serve as a text for an undergraduate course in elementary prob ability theory. The Foundations are presented in: the Introductory Part on the background of the concepts and prob lems, treated without advanced mathematical tools; Part One on the Notions of Measure Theory that every probabilist and statistician requires; Part Two on General Concepts and Tools of Probability Theory. Random sequences whose general properties are given in the Founda tions are studied in: Part Three on Independence devoted essentially to sums of inde pendent random variables and their limit properties; Part Four on Dependence devoted to the operation of conditioning and limit properties of sums of dependent random variables. The last section introduces random functions of second order. Random functions and processes are discussed in: Part Five on Elements of random analysis devoted to the basic con cepts of random analysis and to the martingale, decomposable, and Markov types of random functions. Since the primary purpose of the book is didactic, methods are emphasized and the book is subdivided into: unstarred portions, independent of the remainder; starred portions, which are more involved or more abstract; complements and details, including illustrations and applications of the material in the text, which consist of propositions with fre- PREFACE TO THE THIRD EDITION quent hints; most of these propositions can be found in the articles and books referred to in the Bibliography. Also, for teaching and reference purposes, it has proved useful to name most of the results. Numerous historical remarks about results, methods, and the evolu tion of various fields are an intrinsic part of the text. The purpose is purely didactic: to attract attention to the basic contributions while introducing the ideas explored. Books and memoirs of authors whose contributions are referred to and discussed are cited in the Bibliography, which parallels the text in that it is organized by parts and, within parts, by chapters. Thus the interested student can pursue his study in the original literature. This work owes much to the reactions of the students on whom it has been tried year after year. However, the book is definitely more concise than the lectures, and the reader will have to be armed permanently with patience, pen, and calculus. Besides, in mathematics, as in any form of poetry, the reader has to be a poet in posse. This third edition differs from the second (1960) in a number of places. Modifications vary all the way from a prefix ("sub" martingale in lieu of "semi"-martingale) to an entire subsection (§36.2). To pre serve pagination, some additions to the text proper (especially 9, p. 656) had to be put in the Complements and Details. It is hoped that more over most of the errors have been eliminated and that readers will be kind enough to inform the author of those which remain. I take this opportunity to thank those whose comments and criticisms led to corrections and improvements: for the first edition, E. Barankin, S. Bochner, E. Parzen, and H. Robbins; for the second edition, Y. S. Chow, R. Cogburn, J. L. Doob, J. Feldman, B. Jamison, J. Karush, P. A. Meyer, J. W. Pratt, B. A. Sevastianov, J. W. Wall; for the third edition, S. Dharmadhikari, J. Fabius, D. Freedman, A. Maitra, U. V. Prokhorov. My warm thanks go to Cogburn, whose constant help throughout the preparation of the second edition has been invaluable. This edition has been prepared with the partial support of the Office of Naval Research and of the National Science Foundation. M.L. April, 1962 Berkeley, California CONTENTS OF VOLUME I 45 GRADUATE TEXTS IN MATHEMATICS VOL. INTRODUCTORY PART: ELEMENTARY PROBABILITY THEORY SECTION PAGE I. INTUITIVE BACKGROUND . 3 1. Events . 3 2. Random events and trials 5 3. Random variables 6 II. AxiOMS; INDEPENDENCE AND THE BERNOULLI CASE 8 1. Axioms of the finite case 8 2. Simple random variables 9 3. Independence . 11 4. Bernoulli case . 12 5. Axioms for the countable case 15 6. Elementary random variables 17 7. Need for nonelementary random variables 22 Ill. DEPENDENCE AND CHAINS 24 1. Conditional probabilities 24 2. Asymptotically Bernoullian case 25 3. Recurrence 26 4. Chain dependence 28 *5. Types of states and asymptotic behavior 30 *6. Motion of the system 36 *7. Stationary chains . 39 CoMPLEMENTS AND DETAILS 42 PART ONE: NOTIONS OF MEASURE THEORY CHAPTER I: SET~ SPACES, AND MEASURES 1. SETs, CLAssEs, AND FuNcTIONS • 55 1.1 Definitions and notations 55 1.2 Differences, unions, and intersections 56 1.3 Sequences and limits . 57 1.4 Indicators of sets . 59 xi Xll CONTENTS OF VOLUME I SECTION PAGE 1.5 Fields and u-fields 59 1.6 Monotone classes . 60 *1.7 Product sets . 61 *1.8 Functions and inverse functions 62 *1.9 Measurable spaces and functions 64 *2. ToPOLOGICAL SPACES . 65 *2.1 Topologies and limits 66 *2.2 Limit points and compact spaces 69 *2.3 Countability and metric spaces . 72 *2.4 Linearity and normed spaces 78 3. ADDITIVE SET FuNcTIONS 83 3.1 Additivity and continuity 83 3.2 Decomposition of additive set functions 87 *4. CoNSTRUCTION OF MEASURES ON u-FIELDS 88 *4.1 Extension of measures 88 *4.2 Product probabilities 91 *4.3 Consistent probabilities on Borel fields 93 *4.4 Lebesgue-Stieltjes measures and distribution functions 96 CoMPLEMENTS AND DETAILS 100 CHAPTER II: MEASURABLE FUNCTIONS AND INTEGRATION 5. MEASURABLE FuNCTIONS 103 5.1 Numbers . 103 5.2 Numerical functions 105 5.3 Measurable functions 107 6. MEASURE AND CONVERGENCES 111 6.1 Definitions and general properties Ill 6.2 Convergence almost everywhere 114 6.3 Convergence in measure . 116 7. INTEGRATION . 118 7.1 Integrals 119 7.2 Convergence theorems 125 8. INDEFINITE INTEGRALS; ITERATED INTEGRALS 130 8.1 Indefinite integrals and Lebesgue decomposition 130 8.2 Product measures and iterated integrals . 135 *8.3 Iterated integrals and infinite product spaces . 137 CoMPLEMENTS AND DETAILS 139