Theory of Stochastic Objects Probability, Stochastic Processes and Inference CHAPMAN & HALL/CRC Texts in Statistical Science Series Series Editors Joseph K. Blitzstein, Harvard University, USA Julian J. Faraway, University of Bath, UK Martin Tanner, Northwestern University, USA Jim Zidek, University of British Columbia, Canada Statistical Theory: A Concise Introduction Analysis of Variance, Design, and Regression: F. Abramovich and Y. Ritov Linear Modeling for Unbalanced Data, Second Edition Practical Multivariate Analysis, Fifth Edition A. Afifi, S. May, and V.A. Clark R. Christensen Practical Statistics for Medical Research Bayesian Ideas and Data Analysis: An Introduction D.G. Altman for Scientists and Statisticians R. Christensen, W. Johnson, A. Branscum, Interpreting Data: A First Course and T.E. Hanson in Statistics A.J.B. Anderson Modelling Binary Data, Second Edition D. Collett Introduction to Probability with R K. Baclawski Modelling Survival Data in Medical Research, Third Edition Linear Algebra and Matrix Analysis for Statistics D. Collett S. Banerjee and A. Roy Introduction to Statistical Methods for Modern Data Science with R Clinical Trials B. S. Baumer, D. T. Kaplan, and N. J. Horton T.D. Cook and D.L. DeMets Mathematical Statistics: Basic Ideas and Selected Applied Statistics: Principles and Examples Topics, Volume I, Second Edition P. J. Bickel and K. A. Doksum D.R. Cox and E.J. Snell Mathematical Statistics: Basic Ideas and Selected Multivariate Survival Analysis and Competing Risks Topics, Volume II M. Crowder P. J. Bickel and K. A. Doksum Statistical Analysis of Reliability Data Analysis of Categorical Data with R M.J. Crowder, A.C. Kimber, T.J. Sweeting, C. R. Bilder and T. M. Loughin and R.L. Smith Statistical Methods for SPC and TQM An Introduction to Generalized Linear Models, D. Bissell Third Edition A.J. Dobson and A.G. Barnett Introduction to Probability J. K. Blitzstein and J. Hwang Nonlinear Time Series: Theory, Methods, and Applications with R Examples Bayesian Methods for Data Analysis, Third Edition R. Douc, E. Moulines, and D.S. Stoffer B.P. Carlin and T.A. Louis Introduction to Optimization Methods and Their Statistics in Research and Development, Applications in Statistics Second Edition B.S. Everitt R. Caulcutt Extending the Linear Model with R: Generalized The Analysis of Time Series: An Introduction, Linear, Mixed Effects and Nonparametric Regression Sixth Edition Models, Second Edition C. Chatfield J.J. Faraway Introduction to Multivariate Analysis Linear Models with R, Second Edition C. Chatfield and A.J. Collins J.J. Faraway Problem Solving: A Statistician’s Guide, A Course in Large Sample Theory Second Edition T.S. Ferguson C. Chatfield Multivariate Statistics: A Practical Approach Statistics for Technology: A Course in Applied B. Flury and H. Riedwyl Statistics, Third Edition C. Chatfield Readings in Decision Analysis Exercises and Solutions in Statistical Theory S. French L.L. Kupper, B.H. Neelon, and S.M. O’Brien Discrete Data Analysis with R: Visualization and Design and Analysis of Experiments with R Modeling Techniques for Categorical and Count J. Lawson Data Design and Analysis of Experiments with SAS M. Friendly and D. Meyer J. Lawson Markov Chain Monte Carlo: Stochastic Simulation A Course in Categorical Data Analysis for Bayesian Inference, Second Edition T. Leonard D. Gamerman and H.F. Lopes Statistics for Accountants Bayesian Data Analysis, Third Edition S. Letchford A. Gelman, J.B. Carlin, H.S. Stern, D.B. Dunson, A. Vehtari, and D.B. Rubin Introduction to the Theory of Statistical Inference H. Liero and S. Zwanzig Multivariate Analysis of Variance and Repeated Measures: A Practical Approach for Behavioural Statistical Theory, Fourth Edition Scientists B.W. Lindgren D.J. Hand and C.C. Taylor Stationary Stochastic Processes: Theory and Practical Longitudinal Data Analysis Applications D.J. Hand and M. Crowder G. Lindgren Linear Models and the Relevant Distributions and Statistics for Finance Matrix Algebra E. Lindström, H. Madsen, and J. N. Nielsen D.A. Harville The BUGS Book: A Practical Introduction to Logistic Regression Models Bayesian Analysis J.M. Hilbe D. Lunn, C. Jackson, N. Best, A. Thomas, and D. Spiegelhalter Richly Parameterized Linear Models: Additive, Time Series, and Spatial Models Using Random Effects Introduction to General and Generalized J.S. Hodges Linear Models H. Madsen and P. Thyregod Statistics for Epidemiology N.P. Jewell Time Series Analysis H. Madsen Stochastic Processes: An Introduction, Third Edition P.W. Jones and P. Smith Pólya Urn Models H. Mahmoud The Theory of Linear Models B. Jørgensen Randomization, Bootstrap and Monte Carlo Methods in Biology, Third Edition Pragmatics of Uncertainty B.F.J. Manly J.B. Kadane Statistical Regression and Classification: Principles of Uncertainty From Linear Models to Machine Learning J.B. Kadane N. Matloff Graphics for Statistics and Data Analysis with R Introduction to Randomized Controlled Clinical K.J. Keen Trials, Second Edition Mathematical Statistics J.N.S. Matthews K. Knight Statistical Rethinking: A Bayesian Course with Introduction to Functional Data Analysis Examples in R and Stan P. Kokoszka and M. Reimherr R. McElreath Introduction to Multivariate Analysis: Linear and Statistical Methods in Agriculture and Experimental Nonlinear Modeling Biology, Second Edition S. Konishi R. Mead, R.N. Curnow, and A.M. Hasted Nonparametric Methods in Statistics with SAS Statistics in Engineering: A Practical Approach Applications A.V. Metcalfe O. Korosteleva Theory of Stochastic Objects: Probability, Stochastic Modeling and Analysis of Stochastic Systems, Processes and Inference Third Edition A.C. Micheas V.G. Kulkarni Statistical Inference: An Integrated Approach, Exercises and Solutions in Biostatistical Theory Second Edition L.L. Kupper, B.H. Neelon, and S.M. O’Brien H. S. Migon, D. Gamerman, and F. Louzada Beyond ANOVA: Basics of Applied Statistics Spatio-Temporal Methods in Environmental R.G. Miller, Jr. Epidemiology G. Shaddick and J.V. Zidek A Primer on Linear Models J.F. Monahan Decision Analysis: A Bayesian Approach J.Q. Smith Stochastic Processes: From Applications to Theory P.D Moral and S. Penev Analysis of Failure and Survival Data P. J. Smith Applied Stochastic Modelling, Second Edition B.J.T. Morgan Applied Statistics: Handbook of GENSTAT Analyses Elements of Simulation E.J. Snell and H. Simpson B.J.T. Morgan Applied Nonparametric Statistical Methods, Probability: Methods and Measurement Fourth Edition A. O’Hagan P. Sprent and N.C. Smeeton Introduction to Statistical Limit Theory Data Driven Statistical Methods A.M. Polansky P. Sprent Applied Bayesian Forecasting and Time Series Generalized Linear Mixed Models: Analysis Modern Concepts, Methods and Applications A. Pole, M. West, and J. Harrison W. W. Stroup Statistics in Research and Development, Survival Analysis Using S: Analysis of Time Series: Modeling, Computation, and Inference Time-to-Event Data R. Prado and M. West M. Tableman and J.S. Kim Essentials of Probability Theory for Statisticians Applied Categorical and Count Data Analysis M.A. Proschan and P.A. Shaw W. Tang, H. He, and X.M. Tu Introduction to Statistical Process Control Elementary Applications of Probability Theory, P. Qiu Second Edition Sampling Methodologies with Applications H.C. Tuckwell P.S.R.S. Rao Introduction to Statistical Inference and Its A First Course in Linear Model Theory Applications with R N. Ravishanker and D.K. Dey M.W. Trosset Essential Statistics, Fourth Edition Understanding Advanced Statistical Methods D.A.G. Rees P.H. Westfall and K.S.S. Henning Stochastic Modeling and Mathematical Statistics: Statistical Process Control: Theory and Practice, A Text for Statisticians and Quantitative Scientists Third Edition F.J. Samaniego G.B. Wetherill and D.W. Brown Statistical Methods for Spatial Data Analysis Generalized Additive Models: An Introduction O. Schabenberger and C.A. Gotway with R, Second Edition Bayesian Networks: With Examples in R S. Wood M. Scutari and J.-B. Denis Epidemiology: Study Design and Large Sample Methods in Statistics Data Analysis, Third Edition P.K. Sen and J. da Motta Singer M. Woodward Introduction to Statistical Methods for Financial Practical Data Analysis for Designed Experiments Models B.S. Yandell T. A. Severini Texts in Statistical Science Theory of Stochastic Objects Probability, Stochastic Processes and Inference Athanasios Christou Micheas Department of Statistics, University of Missouri, USA CRC Press Taylor & Francis Group 6000 Broken Sound Parkway NW, Suite 300 Boca Raton, FL 33487-2742 © 2018 by Taylor & Francis Group, LLC CRC Press is an imprint of Taylor & Francis Group, an Informa business No claim to original U.S. Government works Printed on acid-free paper Version Date: 20171219 International Standard Book Number-13: 978-1-4665-1520-8 (Hardback) This book contains information obtained from authentic and highly regarded sources. Reasonable efforts have been made to publish reliable data and information, but the author and publisher cannot assume responsibility for the validity of all materials or the consequences of their use. The authors and publishers have attempted to trace the copyright holders of all material reproduced in this publication and apologize to copyright holders if permission to publish in this form has not been obtained. If any copyright material has not been acknowledged please write and let us know so we may rectify in any future reprint. Except as permitted under U.S. Copyright Law, no part of this book may be reprinted, reproduced, transmitted, or utilized in any form by any electronic, mechanical, or other means, now known or hereafter invented, including photocopying, microfilming, and recording, or in any information storage or retrieval system, without written permission from the publishers. For permission to photocopy or use material electronically from this work, please access www.copyright.com (http://www.copyright.com/) or contact the Copyright Clearance Center, Inc. (CCC), 222 Rosewood Drive, Danvers, MA 01923, 978-750-8400. CCC is a not-for-profit organization that provides licenses and registration for a variety of users. For organizations that have been granted a photocopy license by the CCC, a separate system of payment has been arranged. Trademark Notice: Product or corporate names may be trademarks or registered trademarks, and are used only for identification and explanation without intent to infringe. Library of Congress Cataloging-in-Publication Data Names: Micheas, Athanasios Christou, author. Title: Theory of stochastic objects : probability, stochastic processes and inference / by Athanasios Christou Micheas. Description: Boca Raton, Florida : CRC Press, [2018] | Includes bibliographical references and index. Identifiers: LCCN 2017043053| ISBN 9781466515208 (hardback) | ISBN 9781315156705 (e-book) Subjects: LCSH: Point processes. | Stochastic processes. Classification: LCC QA274.42 .M53 2018 | DDC 519.2/3--dc23 LC record available at https://lccn.loc.gov/2017043053 Visit the Taylor & Francis Web site at http://www.taylorandfrancis.com and the CRC Press Web site at http://www.crcpress.com To my family Contents Preface . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xv List of Figures . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xxi List of Tables . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xxiii List of Abbreviations . . . . . . . . . . . . . . . . . . . . . . . . . . . xxv List of Symbols . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xxvii List of Distribution Notations . . . . . . . . . . . . . . . . . . . . . . xxix 1 Rudimentary Models and Simulation Methods. . . . . . . . . . . 1 1.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1 1.2 RudimentaryProbability . . . . . . . . . . . . . . . . . . . . . . 2 1.2.1 ProbabilityDistributions . . . . . . . . . . . . . . . . . . . 4 1.2.2 Expectation . . . . . . . . . . . . . . . . . . . . . . . . . . 7 1.2.3 MixturesofDistributions . . . . . . . . . . . . . . . . . . 9 1.2.4 TransformationsofRandomVectors . . . . . . . . . . . . . 11 1.3 TheBayesian Approach . . . . . . . . . . . . . . . . . . . . . . 11 1.3.1 Conjugacy . . . . . . . . . . . . . . . . . . . . . . . . . . 11 1.3.2 General PriorSelection MethodsandProperties . . . . . . 14 1.3.3 Hierarchical Bayesian Models . . . . . . . . . . . . . . . . 15 1.4 SimulationMethods . . . . . . . . . . . . . . . . . . . . . . . . 17 1.4.1 TheInverseTransformAlgorithm . . . . . . . . . . . . . . 17 1.4.2 TheAcceptance-Rejection Algorithm . . . . . . . . . . . . 18 1.4.3 TheCompositionMethodforGenerating Mixtures . . . . . 20 1.4.4 GeneratingMultivariateNormaland Related Distributions . 20 1.5 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21 1.6 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24 2 StatisticalInference . . . . . . . . . . . . . . . . . . . . . . . . . . 27 2.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27 2.2 DecisionTheory . . . . . . . . . . . . . . . . . . . . . . . . . . 27 ix