Generalized Linear Models WILEY SERIES IN PROBABILITY AND STATISTICS Established by WALTER A. SHEWHART and SAMUEL S. WILKS Editors: David J Balding, Noel A. C. Cressie, Garrett M. Fitzmaurice, Harvey Goldstein, Iain M. Johnstone, Geert Moienberghs, David W. Scott, Adrian F. M. Smith, Ruey S. Tsay, Sanford Wetsberg Editors Emeriti: Vic Barnett, J. Stuart Hunter, Jozef L. Teugels A complete list of the titles in this series appears at the end of this volume. Generalized Linear Models With Applications in Engineering and the Sciences Second Edition RAYMOND H. MYERS Virginia Polytechnic Institute and State University Blacksburg, Virginia DOUGLAS C. MONTGOMERY Arizona State University Tempe, Arizona G. GEOFFREY VINING Virginia Polytechnic Institute and State University Blacksburg, Virginia TIMOTHY J. ROBINSON University of Wyoming Laramie, Wyoming @ WILEY A JOHN WILEY & SONS, INC., PUBLICATION Copyright © 2010 by John Wiley & Sons, Inc. All rights reserved. Published by John Wiley & Sons, Inc., Hoboken, New Jersey Published simultaneously in Canada No part of this publication may be reproduced, stored in a retrieval system, or transmitted in any form or by any means, electronic, mechanical, photocopying, recording, scanning, or otherwise, except as permitted under Section 107 or 108 of the 1976 United States Copyright Act, without either the prior written permission of the Publisher, or authorization through payment of the appropriate per-copy fee to the Copyright Clearance Center, Inc., 222 Rosewood Drive, Danvers, MA 01923, (978) 750-8400, fax (978) 750-4470, or on the web at www. copyright.com. Requests to the Publisher for permission should be addressed to the Permissions Department, John Wiley & Sons, Inc., Ill River Street, Hoboken, NJ 07030, (201) 748-6011, fax (201) 748-6008, or online at http://www.wiley.com/go/permission. Limit of Liability/Disclaimer of Warranty: While the publisher and author have used their best efforts in preparing this book, they make no representations or warranties with respect to the accuracy or completeness of the contents of this book and specifically disclaim any implied warranties of merchantability or fitness for a particular purpose. No warranty may be created or extended by sales representatives or written sales materials. The advice and strategies contained herein may not be suitable for your situation. You should consult with a professional where appropriate. Neither the publisher nor author shall be liable for any loss of profit or any other commercial damages, including but not limited to special, incidental, consequential, or other damages. For general information on our other products and services or for technical support, please contact our Customer Care Department within the United States at (800) 762-2974, outside the United States at (317) 572-3993 or fax (317) 572-4002. Wiley also publishes its books in a variety of electronic formats. Some content that appears in print may not be available in electronic formats. For more information about Wiley products, visit our web site at www.wiley.com. Library of Congress Cataloging-in-Publication Data: Generalized linear models : with applications in engineering and the sciences / Raymond H. Myers ... [et al.]. — 2nd ed. p. cm. Rev. ed. of: Generalized linear models / Raymond H. Myers, Douglas C. Montgomery, G. Geoffrey Vining. c2002. Includes bibliographical references and index. ISBN 978-0-470-45463-3 (cloth) 1. Linear models (Statistics) I. Myers, Raymond H. Generalized linear models. QA276.M94 2010 519.5'35—dc22 2009049310 Printed in the United States of America 10 9 8 7 6 5 4 3 21 Contents Preface xi 1. Introduction to Generalized Linear Models 1 1.1 Linear Models, 1 1.2 Nonlinear Models, 3 1.3 The Generalized Linear Model, 4 2. Linear Regression Models 9 2.1 The Linear Regression Model and Its Application, 9 2.2 Multiple Regression Models, 10 2.2.1 Parameter Estimation with Ordinary Least Squares, 10 2.2.2 Properties of the Least Squares Estimator and Estimation of σ2, 15 2.2.3 Hypothesis Testing in Multiple Regression, 19 2.2.4 Confidence Intervals in Multiple Regression, 29 2.2.5 Prediction of New Response Observations, 32 2.2.6 Linear Regression Computer Output, 34 2.3 Parameter Estimation Using Maximum Likelihood, 34 2.3.1 Parameter Estimation Under the Normal-Theory Assumptions, 34 2.3.2 Properties of the Maximum Likelihood Estimators, 38 2.4 Model Adequacy Checking, 39 2.4.1 Residual Analysis, 39 2.4.2 Transformation of the Response Variable Using the Box-Cox Method, 43 v 2.4.3 Scaling Residuals, 45 2.4.4 Influence Diagnostics, 50 2.5 Using R to Perform Linear Regression Analysis, 52 2.6 Parameter Estimation by Weighted Least Squares, 54 2.6.1 The Constant Variance Assumption, 54 2.6.2 Generalized and Weighted Least Squares, 55 2.6.3 Generalized Least Squares and Maximum Likelihood, 58 2.7 Designs for Regression Models, 58 Exercises, 65 3. Nonlinear Regression Models 3.1 Linear and Nonlinear Regression Models, 77 3.1.1 Linear Regression Models, 77 3.1.2 Nonlinear Regression Models, 78 3.1.3 Origins of Nonlinear Models, 79 3.2 Transforming to a Linear Model, 81 3.3 Parameter Estimation in a Nonlinear System, 84 3.3.1 Nonlinear Least Squares, 84 3.3.2 The Geometry of Linear and Nonlinear Least Squares, 86 3.3.3 Maximum Likelihood Estimation, 86 3.3.4 Linearization and the Gauss-Newton Method, 89 3.3.5 Using R to Perform Nonlinear Regression Analysis, 99 3.3.6 Other Parameter Estimation Methods, 100 3.3.7 Starting Values, 101 3.4 Statistical Inference in Nonlinear Regression, 102 3.5 Weighted Nonlinear Regression, 106 3.6 Examples of Nonlinear Regression Models, 107 3.7 Designs for Nonlinear Regression Models, 108 Exercises, 111 4. Logistic and Poisson Regression Models 4.1 Regression Models Where the Variance Is a Function of the Mean, 119 4.2 Logistic Regression Models, 120 4.2.1 Models with a Binary Response Variable, 120 4.2.2 Estimating the Parameters in a Logistic Regression Model, 123 CONTENTS vii 4.2.3 Interpertation of the Parameters in a Logistic Regression Model, 128 4.2.4 Statistical Inference on Model Parameters, 132 4.2.5 Lack-of-Fit Tests in Logistic Regression, 143 4.2.6 Diagnostic Checking in Logistic Regression, 155 4.2.7 Classification and the Receiver Operating Characteristic Curve, 162 4.2.8 A Biological Example of Logistic Regression, 164 4.2.9 Other Models for Binary Response Data, 168 4.2.10 More than Two Categorical Outcomes, 169 4.3 Poisson Regression, 176 4.4 Overdispersion in Logistic and Poisson Regression, 184 Exercises, 189 5. The Generalized Linear Model 202 5.1 The Exponential Family of Distributions, 202 5.2 Formal Structure for the Class of Generalized Linear Models, 205 5.3 Likelihood Equations for Generalized Linear models, 207 5.4 Quasi-Likelihood, 211 5.5 Other Important Distributions for Generalized Linear Models, 213 5.5.1 The Gamma Family, 214 5.5.2 Canonical Link Function for the Gamma Distribution, 215 5.5.3 Log Link for the Gamma Distribution, 215 5.6 A Class of Link Functions—The Power Function, 216 5.7 Inference and Residual Analysis for Generalized Linear Models, 217 5.8 Examples with the Gamma Distribution, 220 5.9 Using R to Perform GLM Analysis, 229 5.9.1 Logistic Regression, Each Response is a Success or Failure, 231 5.9.2 Logistic Regression, Response is the Number of Successes Out of n Trials, 232 5.9.3 Poisson Regression, 232 5.9.4 Using the Gamma Distribution with a Log Link, 233 5.10 GLM and Data Transformation, 233 viii CONTENTS 5.11 Modeling Both a Process Mean and Process Variance Using GLM, 240 5.11.1 The Replicated Case, 240 5.11.2 The Unreplicated Case, 244 5.12 Quality of Asymptotic Results and Related Issues, 250 5.12.1 Development of an Alternative Wald Confidence Interval, 250 5.12.2 Estimation of Exponential Family Scale Parameter, 259 5.12.3 Impact of Link Misspecification on Confidence Interval Coverage and Precision, 260 5.12.4 Illustration of Binomial Distribution with a True Identity Link but with Logit Link Assumed, 260 5.12.5 Poisson Distribution with a True Identity Link but with Log Link Assumed, 262 5.12.6 Gamma Distribution with a True Inverse Link but with Log Link Assumed, 263 5.12.7 Summary of Link Misspecification on Confidence Interval Coverage and Precision, 264 5.12.8 Impact of Model Misspecification on Confidence Interval Coverage and Precision, 264 Exercises, 267 6. Generalized Estimating Equations 272 6.1 Data Layout for Longitudinal Studies, 272 6.2 Impact of the Correlation Matrix R, 274 6.3 Iterative Procedure in the Normal Case, Identity Link, 275 6.4 Generalized Estimating Equations for More Generalized Linear Models, 277 6.4.1 Structure of V , 278 7 6.4.2 Iterative Computation of Elements in R, 283 6.5 Examples, 283 6.6 Summary, 308 Exercises, 311 7. Random Effects in Generalized Linear Models 319 7.1 Linear Mixed Effects Models, 320 7.1.1 Linear Regression Models, 320 7.1.2 General Linear Mixed Effects Models, 322 7.1.3 Covariance Matrix, V, 326