Bayesian Networks With Examples in R K22427_FM.indd 1 5/14/14 3:43 PM CHAPMAN & HALL/CRC Texts in Statistical Science Series Series Editors Francesca Dominici, Harvard School of Public Health, USA Julian J. Faraway, University of Bath, UK Martin Tanner, Northwestern University, USA Jim Zidek, University of British Columbia, Canada Statistical Theory: A Concise Introduction Introduction to Statistical Methods for F. Abramovich and Y. Ritov Clinical Trials T.D. Cook and D.L. DeMets Practical Multivariate Analysis, Fifth Edition A. Afifi, S. May, and V.A. Clark Applied Statistics: Principles and Examples Practical Statistics for Medical Research D.R. Cox and E.J. Snell D.G. Altman Multivariate Survival Analysis and Competing Interpreting Data: A First Course Risks in Statistics M. Crowder A.J.B. Anderson Statistical Analysis of Reliability Data Introduction to Probability with R M.J. Crowder, A.C. Kimber, K. Baclawski T.J. Sweeting, and R.L. Smith Linear Algebra and Matrix Analysis for An Introduction to Generalized Statistics Linear Models, Third Edition S. Banerjee and A. Roy A.J. Dobson and A.G. Barnett Statistical Methods for SPC and TQM Nonlinear Time Series: Theory, Methods, and D. Bissell Applications with R Examples R. Douc, E. Moulines, and D.S. Stoffer Bayesian Methods for Data Analysis, Third Edition Introduction to Optimization Methods and B.P. Carlin and T.A. Louis Their Applications in Statistics B.S. Everitt Second Edition R. Caulcutt Extending the Linear Model with R: Generalized Linear, Mixed Effects and The Analysis of Time Series: An Introduction, Nonparametric Regression Models Sixth Edition J.J. Faraway C. Chatfield Linear Models with R, Second Edition Introduction to Multivariate Analysis J.J. Faraway C. Chatfield and A.J. Collins A Course in Large Sample Theory Problem Solving: A Statistician’s Guide, T.S. Ferguson Second Edition C. Chatfield Multivariate Statistics: A Practical Approach Statistics for Technology: A Course in Applied B. Flury and H. Riedwyl Statistics, Third Edition C. Chatfield Readings in Decision Analysis S. French Bayesian Ideas and Data Analysis: An Introduction for Scientists and Statisticians Markov Chain Monte Carlo: Stochastic Simulation for Bayesian Inference, R. Christensen, W. Johnson, A. Branscum, Second Edition and T.E. Hanson D. Gamerman and H.F. Lopes Modelling Binary Data, Second Edition Bayesian Data Analysis, Third Edition D. Collett A. Gelman, J.B. Carlin, H.S. Stern, D.B. Dunson, Modelling Survival Data in Medical Research, A. Vehtari, and D.B. Rubin Second Edition D. Collett K22427_FM.indd 2 5/14/14 3:43 PM Multivariate Analysis of Variance and Statistical Theory, Fourth Edition Repeated Measures: A Practical Approach for B.W. Lindgren Behavioural Scientists Stationary Stochastic Processes: Theory and D.J. Hand and C.C. Taylor Applications Practical Data Analysis for Designed Practical G. Lindgren Longitudinal Data Analysis The BUGS Book: A Practical Introduction to D.J. Hand and M. Crowder Bayesian Analysis Logistic Regression Models D. Lunn, C. Jackson, N. Best, A. Thomas, and J.M. Hilbe D. Spiegelhalter Richly Parameterized Linear Models: Introduction to General and Generalized Additive, Time Series, and Spatial Models Linear Models Using Random Effects H. Madsen and P. Thyregod J.S. Hodges Time Series Analysis Statistics for Epidemiology H. Madsen N.P. Jewell Pólya Urn Models Stochastic Processes: An Introduction, H. Mahmoud Second Edition Randomization, Bootstrap and Monte Carlo P.W. Jones and P. Smith Methods in Biology, Third Edition The Theory of Linear Models B.F.J. Manly B. Jørgensen Introduction to Randomized Controlled Principles of Uncertainty Clinical Trials, Second Edition J.B. Kadane J.N.S. Matthews Graphics for Statistics and Data Analysis with R Statistical Methods in Agriculture and K.J. Keen Experimental Biology, Second Edition R. Mead, R.N. Curnow, and A.M. Hasted Mathematical Statistics K. Knight Statistics in Engineering: A Practical Approach A.V. Metcalfe Introduction to Multivariate Analysis: Linear and Nonlinear Modeling Beyond ANOVA: Basics of Applied Statistics S. Konishi R.G. Miller, Jr. Nonparametric Methods in Statistics with SAS A Primer on Linear Models Applications J.F. Monahan O. Korosteleva Applied Stochastic Modelling, Second Edition Modeling and Analysis of Stochastic Systems, B.J.T. Morgan Second Edition Elements of Simulation V.G. Kulkarni B.J.T. Morgan Exercises and Solutions in Biostatistical Theory Probability: Methods and Measurement L.L. Kupper, B.H. Neelon, and S.M. O’Brien A. O’Hagan Exercises and Solutions in Statistical Theory Introduction to Statistical Limit Theory L.L. Kupper, B.H. Neelon, and S.M. O’Brien A.M. Polansky Design and Analysis of Experiments with SAS Applied Bayesian Forecasting and Time Series J. Lawson Analysis A Course in Categorical Data Analysis A. Pole, M. West, and J. Harrison T. Leonard Statistics in Research and Development, Statistics for Accountants Time Series: Modeling, Computation, and S. Letchford Inference Introduction to the Theory of Statistical R. Prado and M. West Inference Introduction to Statistical Process Control H. Liero and S. Zwanzig P. Qiu K22427_FM.indd 3 5/14/14 3:43 PM Sampling Methodologies with Applications Generalized Linear Mixed Models: P.S.R.S. Rao Modern Concepts, Methods and Applications W. W. Stroup A First Course in Linear Model Theory N. Ravishanker and D.K. Dey Survival Analysis Using S: Analysis of Time-to-Event Data Essential Statistics, Fourth Edition M. Tableman and J.S. Kim D.A.G. Rees Applied Categorical and Count Data Analysis Stochastic Modeling and Mathematical W. Tang, H. He, and X.M. Tu Statistics: A Text for Statisticians and Quantitative Elementary Applications of Probability Theory, F.J. Samaniego Second Edition H.C. Tuckwell Statistical Methods for Spatial Data Analysis O. Schabenberger and C.A. Gotway Introduction to Statistical Inference and Its Applications with R Bayesian Networks: With Examples in R M.W. Trosset M. Scutari and J.-B. Denis Understanding Advanced Statistical Methods Large Sample Methods in Statistics P.H. Westfall and K.S.S. Henning P.K. Sen and J. da Motta Singer Statistical Process Control: Theory and Decision Analysis: A Bayesian Approach Practice, Third Edition J.Q. Smith G.B. Wetherill and D.W. Brown Analysis of Failure and Survival Data Generalized Additive Models: P. J. Smith An Introduction with R Applied Statistics: Handbook of GENSTAT S. Wood Analyses Epidemiology: Study Design and E.J. Snell and H. Simpson Data Analysis, Third Edition Applied Nonparametric Statistical Methods, M. Woodward Fourth Edition Experiments P. Sprent and N.C. Smeeton B.S. Yandell Data Driven Statistical Methods P. Sprent K22427_FM.indd 4 5/14/14 3:43 PM Texts in Statistical Science Bayesian Networks With Examples in R Marco Scutari UCL Genetics Institute (UGI) London, United Kingdom Jean-Baptiste Denis Unité de Recherche Mathématiques et Informatique Appliquées, INRA, Jouy-en-Josas, France K22427_FM.indd 5 5/14/14 3:43 PM CRC Press Taylor & Francis Group 6000 Broken Sound Parkway NW, Suite 300 Boca Raton, FL 33487-2742 © 2015 by Taylor & Francis Group, LLC CRC Press is an imprint of Taylor & Francis Group, an Informa business No claim to original U.S. Government works Version Date: 20140514 International Standard Book Number-13: 978-1-4822-2559-4 (eBook - PDF) This book contains information obtained from authentic and highly regarded sources. Reasonable efforts have been made to publish reliable data and information, but the author and publisher cannot assume responsibility for the validity of all materials or the consequences of their use. The authors and publishers have attempted to trace the copyright holders of all material reproduced in this publication and apologize to copyright holders if permission to publish in this form has not been obtained. If any copyright material has not been acknowledged please write and let us know so we may rectify in any future reprint. Except as permitted under U.S. Copyright Law, no part of this book may be reprinted, reproduced, transmitted, or utilized in any form by any electronic, mechanical, or other means, now known or hereafter invented, including photocopying, microfilming, and recording, or in any information stor- age or retrieval system, without written permission from the publishers. For permission to photocopy or use material electronically from this work, please access www.copy- right.com (http://www.copyright.com/) or contact the Copyright Clearance Center, Inc. (CCC), 222 Rosewood Drive, Danvers, MA 01923, 978-750-8400. CCC is a not-for-profit organization that pro- vides licenses and registration for a variety of users. For organizations that have been granted a pho- tocopy license by the CCC, a separate system of payment has been arranged. Trademark Notice: Product or corporate names may be trademarks or registered trademarks, and are used only for identification and explanation without intent to infringe. Visit the Taylor & Francis Web site at http://www.taylorandfrancis.com and the CRC Press Web site at http://www.crcpress.com To my family To my wife, Jeanie Denis Contents Preface xiii 1 The Discrete Case: Multinomial Bayesian Networks 1 1.1 Introductory Example: Train Use Survey . . . . . . . . . . . 1 1.2 Graphical Representation . . . . . . . . . . . . . . . . . . . . 2 1.3 Probabilistic Representation . . . . . . . . . . . . . . . . . . 7 1.4 Estimating the Parameters: Conditional Probability Tables . 11 1.5 Learning the DAG Structure: Tests and Scores . . . . . . . . 14 1.5.1 Conditional Independence Tests. . . . . . . . . . . . . 15 1.5.2 Network Scores . . . . . . . . . . . . . . . . . . . . . . 17 1.6 Using Discrete BNs . . . . . . . . . . . . . . . . . . . . . . . 20 1.6.1 Using the DAG Structure . . . . . . . . . . . . . . . . 20 1.6.2 Using the Conditional Probability Tables . . . . . . . 23 1.6.2.1 Exact Inference . . . . . . . . . . . . . . . . 23 1.6.2.2 Approximate Inference . . . . . . . . . . . . 27 1.7 Plotting BNs . . . . . . . . . . . . . . . . . . . . . . . . . . . 29 1.7.1 Plotting DAGs . . . . . . . . . . . . . . . . . . . . . . 29 1.7.2 Plotting Conditional Probability Distributions . . . . 31 1.8 Further Reading . . . . . . . . . . . . . . . . . . . . . . . . . 33 2 The Continuous Case: Gaussian Bayesian Networks 37 2.1 Introductory Example: Crop Analysis . . . . . . . . . . . . . 37 2.2 Graphical Representation . . . . . . . . . . . . . . . . . . . . 38 2.3 Probabilistic Representation . . . . . . . . . . . . . . . . . . 42 2.4 Estimating the Parameters: Correlation Coefficients . . . . . 46 2.5 Learning the DAG Structure: Tests and Scores . . . . . . . . 49 2.5.1 Conditional Independence Tests. . . . . . . . . . . . . 49 2.5.2 Network Scores . . . . . . . . . . . . . . . . . . . . . . 52 2.6 Using Gaussian Bayesian Networks . . . . . . . . . . . . . . 52 2.6.1 Exact Inference . . . . . . . . . . . . . . . . . . . . . . 53 2.6.2 Approximate Inference . . . . . . . . . . . . . . . . . . 54 2.7 Plotting Gaussian Bayesian Networks . . . . . . . . . . . . . 57 2.7.1 Plotting DAGs . . . . . . . . . . . . . . . . . . . . . . 57 2.7.2 Plotting Conditional Probability Distributions . . . . 59 ix