UUnniivveerrssiittyy ooff NNeebbrraasskkaa -- LLiinnccoollnn DDiiggiittaallCCoommmmoonnss@@UUnniivveerrssiittyy ooff NNeebbrraasskkaa -- LLiinnccoollnn Dissertations and Theses in Statistics Statistics, Department of 12-2009 FFUULLLLYY EEXXPPOONNEENNTTIIAALL LLAAPPLLAACCEE AAPPPPRROOXXIIMMAATTIIOONN EEMM AALLGGOORRIITTHHMM FFOORR NNOONNLLIINNEEAARR MMIIXXEEDD EEFFFFEECCTTSS MMOODDEELLSS Meijian Zhou University of Nebraska-Lincoln, [email protected] Follow this and additional works at: https://digitalcommons.unl.edu/statisticsdiss Part of the Statistical Methodology Commons Zhou, Meijian, "FULLY EXPONENTIAL LAPLACE APPROXIMATION EM ALGORITHM FOR NONLINEAR MIXED EFFECTS MODELS" (2009). Dissertations and Theses in Statistics. 4. https://digitalcommons.unl.edu/statisticsdiss/4 This Article is brought to you for free and open access by the Statistics, Department of at DigitalCommons@University of Nebraska - Lincoln. It has been accepted for inclusion in Dissertations and Theses in Statistics by an authorized administrator of DigitalCommons@University of Nebraska - Lincoln. FULLY EXPONENTIAL LAPLACE APPROXIMATION EM ALGORITHM FOR NONLINEAR MIXED EFFECTS MODELS by Meijian Zhou A DISSERTATION Presented to the Faculty of The Graduate College at the University of Nebraska In Partial Fulfillment of Requirements For the Degree of Doctor of Philosophy Major: Statistics Under the Supervision of Professor Anne M. Parkhurst Lincoln, Nebraska December, 2009 FULLY EXPONENTIAL LAPLACE APPROXIMATION EM ALGORITHM FOR NONLINEAR MIXED EFFECTS MODELS Meijian Zhou, Ph.D. University of Nebraska, 2009 Advisor: Anne M. Parkhurst Nonlinear mixed effects models provide a flexible and powerful platform for the analysis of clustered data that arise in numerous fields, such as pharmacology, biology, agriculture, forestry, and economics. This dissertation focuses on fitting parametric nonlinear mixed effects models with single- and multi-level random effects. A new, efficient, and accurate method that gives an error of order O(1/n2) , fully exponential Laplace approximation EM algorithm (FELA-EM), for obtaining restricted maximum likelihood (REML) estimates in nonlinear mixed effects models is developed. Sample codes for implementing FELA-EM algorithm in R are given. Simulation studies have been conducted to evaluate the accuracy of the new approach and compare it with the Laplace approximation as well as four different linearization methods for fitting nonlinear mixed effects models with single-level and two-crossed-level random effects. Of all approximations considered in the thesis, FELA-EM algorithm is the only one that gives unbiased or close-to-unbiased (%Bias < 1%) estimates for both the fixed effects and variance-covariance parameters. Finally, FELA-EM algorithm is applied to a real dataset to model feeding pigs’ body temperature and a unified strategy for building crossed and nested nonlinear mixed effects models with treatments and covariates is provided. iii ACKNOWLEDGEMENTS I would like to thank my advisor, Dr. Anne M. Parkhurst, for offering me a graduate research assistantship [1,2,3] and for all her inspiring guidance, full support and patience throughout this research project. I would like to thank my supervisory committee, Dr. Kent M. Eskridge and Dr. Shunpu Zhang for giving helpful advice and perceptive comments on my dissertation, and Dr. Roger A. Eigenberg for sharing his data and helping me understand his research and experiments on heat dissipation in swine. I would like to thank B. C. Pollard and Dr. Robert J. Collier from University of Arizona and Dr. John A. Nienaber and Dr. G. Leroy Hahn from U.S. Meat Animal Research Center for sharing their data to allow me to pursue my interest in applications of nonlinear mixed effects models. I would also like to thank Dr. David A. Fournier for helping me develop codes for crossed and nested nonlinear mixed models in ADMB-RE and Dr. Oliver Schabenberger for providing valuable advice on fitting nonlinear mixed models with both crossed and nested random effects in the SAS macro %NLINMIX. Special thanks to my wife Hui Shen, my daughters, Amy Zhou and Claire Zhou, my parents, Mr. Yiben Zhou and Mrs. Yuehua Zhu, my sisters, Guangqin Zhou and Xueqin Zhou and their families, for their love and support during the five years of study. Reference [1] U.S. Department of Agriculture, Agricultural Research Division [2] Institute of Agriculture and Natural Resources, University of Nebraska-Lincoln [3] Multi-State Research Project W173: Stress Factors of Farm Animals and Their Effects on Performance iv TABLE OF CONTENTS Acknowledgments............................................................................................................. iii Table of Contents ............................................................................................................... iv List of Figures ................................................................................................................. viii List of Tables ..................................................................................................................... ix 1 Introduction ................................................................................................................. 1 1.1 Motivation ............................................................................................................ 1 1.2 Summary of the remaining chapters .................................................................... 4 1.3 References ............................................................................................................ 5 2 Literature Review ....................................................................................................... 8 2.1 Introduction .......................................................................................................... 8 2.2 Linear mixed effects models .............................................................................. 10 2.3 Nonlinear regression models.............................................................................. 11 2.4 Nonlinear mixed effects models ........................................................................ 12 2.5 Four categories of nonlinear mixed effects models ........................................... 13 2.5.1 Parametric nonlinear mixed effects models ........................................... 13 2.5.2 Nonparametric nonlinear mixed effects models .................................... 14 2.5.3 Semi-parametric nonlinear mixed effects models .................................. 15 2.5.4 Bayesian approach to nonlinear mixed effects models .......................... 15 v 2.6 Estimation methods of parametric nonlinear mixed effects models .................. 16 2.6.1 Linearization methods ............................................................................ 17 2.6.2 Integral approximation methods ............................................................ 19 2.6.3 EM algorithms ....................................................................................... 24 2.7 Software review ................................................................................................. 27 2.8 References .......................................................................................................... 32 3 REML Estimation in Nonlinear Mixed Effects Models via the Fully Exponential Laplace Approximation EM Algorithm .................................................................. 41 3.0 Abstract .............................................................................................................. 41 3.1 Introduction ........................................................................................................ 42 3.2 Model and likelihood ......................................................................................... 45 3.3 FELA-EM algorithm for REML estimates of variance-covariance components ........................................................................................................ 47 3.3.1 E-step ..................................................................................................... 48 3.3.2 M-step .................................................................................................... 50 3.3.3 Fully exponential Laplace approximation ............................................. 51 3.3.4 Calculating the information matrix ........................................................ 56 3.3.5 Estimating the fixed and random effects ............................................... 57 3.4 Comparing the approximations .......................................................................... 58 3.4.1 Logistic model ....................................................................................... 59 3.4.2 First-order compartment model ............................................................. 71 3.5 Discussion .......................................................................................................... 81 vi 3.6 Conclusions ........................................................................................................ 83 3.7 Summary ............................................................................................................ 85 3.8 References .......................................................................................................... 86 4 Extension of the Fully Exponential Laplace Approximation EM Algorithm for Nonlinear Mixed Models with two Levels of Crossed Random Effects ............... 90 4.0 Abstract .............................................................................................................. 90 4.1 Introduction ........................................................................................................ 92 4.2 Model and likelihood ......................................................................................... 95 4.3 Laplace approximation to the likelihood ........................................................... 97 4.4 FELA-EM algorithm ........................................................................................ 100 4.4.1 E-step ................................................................................................... 101 4.4.2 M-step .................................................................................................. 104 4.4.3 Fully exponential Laplace approximation ........................................... 106 4.4.4 Calculating the information matrix ...................................................... 111 4.4.5 Estimating the fixed and random effects ............................................. 111 4.5 Comparing the approximations ........................................................................ 112 4.5.1 Logistic model ..................................................................................... 112 4.5.2 First-order compartment model ........................................................... 122 4.6 Discussion ........................................................................................................ 129 4.7 Conclusions ...................................................................................................... 131 4.8 Summary .......................................................................................................... 132 4.9 References ........................................................................................................ 133 vii 5 Multilevel Nonlinear Mixed Effects Models with both Crossed and Nested Random Effects Applied in a Replicated Latin Square Design for Modeling Temperature of Feeding Pigs ................................................................................. 137 5.0 Abstract ............................................................................................................ 137 5.1 Introduction ...................................................................................................... 138 5.2 Materials and methods ..................................................................................... 141 5.2.1 Data ...................................................................................................... 141 5.2.2 Statistical model ................................................................................... 144 5.2.3 Crossed and nested random effects ...................................................... 146 5.2.4 Model building ..................................................................................... 146 5.2.5 Review of FELA-EM algorithm .......................................................... 149 5.3 Results and discussion ..................................................................................... 155 5.3.1 Specification of random effects ........................................................... 155 5.3.2 Specification of within-event error correlation structure ..................... 157 5.3.3 Model diagnostics ................................................................................ 158 5.3.4 Comparison of the three thermal environmental treatments and test of the feed intake and meal duration effects ........................... 162 5.4 Conclusions ...................................................................................................... 165 5.5 Summary .......................................................................................................... 166 5.6 References ........................................................................................................ 167 Appendix: R program for fitting the logistic model formulated by the equation (4.4.1) using FELA-EM algorithm ........................................................................ 171 viii LIST OF FIGURES 3.1 Example of simulated logistic curves for two settings of variance and covariance parameters .............................................................................................. 60 3.2 Example of simulated first-order compartment curves for two settings of variance and covariance parameters .................................................................... 72 5.1 Example of changes in tympanic temperature (ºC) and feed intake (kg) of pigs over Julian calendar time for pig 27 (a member of the heavy group) during first experimental period in the second run under treatment 2 (28ºC and high air speed) .................................................................... 143 5.2 Autocorrelation function corresponding to the within-event errors of Model 7 .................................................................................................................. 157 5.3 Autocorrelation function corresponding to the within-event errors of Model 9 .................................................................................................................. 157 5.4 Scatter plot of standardized residuals versus fitted values for Model 9 ................ 160 5.5 Normal plot of standardized residuals for Model 9 ............................................... 160 5.6 Normal plot of the estimated random effects for Model 9 ..................................... 160 5.7 Observed (○) and predicted (―) tympanic temperatures (C) over time (min) for eighteen feeding events .................................................................. 161 ix LIST OF TABLES 3.1 Simulation results for the fixed effects in the logistic model for small D and σ2 ...................................................................................................... 64 3.2 Simulation results for the fixed effects in the logistic model for large D and σ2 ....................................................................................................... 65 3.3 Simulation results for the variance-covariance components in the logistic model for small D and σ2 ......................................................................... 69 3.4 Simulation results for the variance-covariance components in the logistic model for large D and σ2 ......................................................................... 70 3.5 Simulation results for the fixed effects in the first-order compartment model for small D and σ2 ..................................................................................... 75 3.6 Simulation results for the fixed effects in the first-order compartment model for large D and σ2 ...................................................................................... 76 3.7 Simulation results for the variance-covariance components in the first-order compartment model for small D and σ2 .............................................. 79 3.8 Simulation results for the variance-covariance components in the first-order compartment model for large D and σ2 ............................................... 80 4.1 Simulation results for the fixed effects in the logistic model ................................ 117 4.2 Simulation results for the variance-covariance components in the logistic model ......................................................................................................... 120