The UNDERPGRuAreD UanATdE A p TpEliXeTdS SERIES Sally Foundations and Applications of Statistics An Introduction Using R Randall Pruim American Mathematical Society Foundations and Applications of Statistics An Introduction Using R T h e UNDERGRADUATE TEXTS • 13 SERIES Pure and Applied Sally Foundations and Applications of Statistics An Introduction Using R Randall Pruim American Mathematical Society Providence, Rhode Island EDITORIAL COMMITTEE Paul J. Sally, Jr. (Chair) Joseph Silverman Francis Su Susan Tolman 2010 Mathematics Subject Classification. Primary 62–01; Secondary 60–01. For additional informationand updates on this book, visit www.ams.org/bookpages/amstext-13 Library of Congress Cataloging-in-Publication Data Pruim,RandallJ. Foundationsandapplicationsofstatistics: anintroductionusingR/RandallPruim. p.cm. —(Pureandappliedundergraduatetexts;v.13) Includesbibliographicalreferencesandindex. ISBN978-0-8218-5233-0(alk.paper) 1.Mathematicalstatistics—Dataprocessing. 2.R(Computerprogramlanguage) I.Title. QA276.45.R3P78 2010 519.50285—dc22 2010041197 Copying and reprinting. Individual readers of this publication, and nonprofit libraries actingforthem,arepermittedtomakefairuseofthematerial,suchastocopyachapterforuse in teaching or research. Permission is granted to quote brief passages from this publication in reviews,providedthecustomaryacknowledgmentofthesourceisgiven. Republication,systematiccopying,ormultiplereproductionofanymaterialinthispublication is permitted only under license from the American Mathematical Society. Requests for such permissionshouldbeaddressedtotheAcquisitionsDepartment,AmericanMathematicalSociety, 201 Charles Street, Providence, Rhode Island 02904-2294 USA. Requests can also be made by [email protected]. (cid:2)c 2011byRandallPruim. Allrightsreserved. PrintedintheUnitedStatesofAmerica. (cid:2)∞ Thepaperusedinthisbookisacid-freeandfallswithintheguidelines establishedtoensurepermanenceanddurability. VisittheAMShomepageathttp://www.ams.org/ 10987654321 161514131211 Contents Preface ix What Is Statistics? xv Chapter 1. Summarizing Data 1 1.1. Data in R 1 1.2. Graphical and Numerical Summaries of Univariate Data 4 1.3. Graphical and Numerical Summaries of Multivariate Data 17 1.4. Summary 20 Exercises 21 Chapter 2. Probability and Random Variables 27 2.1. Introduction to Probability 28 2.2. Additional Probability Rules and Counting Methods 33 2.3. Discrete Distributions 49 2.4. Hypothesis Tests and p-Values 56 2.5. Mean and Variance of a Discrete Random Variable 65 2.6. Joint Distributions 72 2.7. Other Discrete Distributions 80 2.8. Summary 92 Exercises 97 Chapter 3. Continuous Distributions 113 3.1. pdfs and cdfs 113 3.2. Mean and Variance 124 3.3. Higher Moments 126 3.4. Other Continuous Distributions 133 v vi Contents 3.5. Kernel Density Estimation 145 3.6. Quantile-Quantile Plots 150 3.7. Joint Distributions 155 3.8. Summary 163 Exercises 166 Chapter 4. Parameter Estimation and Testing 175 4.1. Statistical Models 175 4.2. Fitting Models by the Method of Moments 177 4.3. Estimators and Sampling Distributions 183 4.4. Limit Theorems 191 4.5. Inference for the Mean (Variance Known) 200 4.6. Estimating Variance 208 4.7. Inference for the Mean (Variance Unknown) 212 4.8. Confidence Intervals for a Proportion 223 4.9. Paired Tests 225 4.10. Developing New Tests 227 4.11. Summary 234 Exercises 238 Chapter 5. Likelihood-Based Statistics 251 5.1. Maximum Likelihood Estimators 251 5.2. Likelihood Ratio Tests 266 5.3. Confidence Intervals 274 5.4. Goodness of Fit Testing 277 5.5. Inference for Two-Way Tables 288 5.6. Rating and Ranking Based on Pairwise Comparisons 297 5.7. Bayesian Inference 304 5.8. Summary 312 Exercises 315 Chapter 6. Introduction to Linear Models 323 6.1. The Linear Model Framework 324 6.2. Simple Linear Regression 333 6.3. Inference for Simple Linear Regression 341 6.4. Regression Diagnostics 353 6.5. Transformations in Linear Regression 362 6.6. Categorical Predictors 369 6.7. Categorical Response (Logistic Regression) 377 6.8. Simulating Linear Models to Check Robustness 384 Contents vii 6.9. Summary 387 Exercises 391 Chapter 7. More Linear Models 399 7.1. Additive Models 399 7.2. Assessing the Quality of a Model 413 7.3. One-Way ANOVA 423 7.4. Two-Way ANOVA 455 7.5. Interaction and Higher Order Terms 470 7.6. Model Selection 474 7.7. More Examples 482 7.8. Permutation Tests and Linear Models 493 7.9. Summary 496 Exercises 499 Appendix A. A Brief Introduction to R 507 A.1. Getting Up and Running 507 A.2. Working with Data 513 A.3. Lattice Graphics in R 527 A.4. Functions in R 535 A.5. Some Extras in the fastR Package 542 A.6. More R Topics 544 Exercises 546 Appendix B. Some Mathematical Preliminaries 549 B.1. Sets 550 B.2. Functions 552 B.3. Sums and Products 553 Exercises 555 Appendix C. Geometry and Linear Algebra Review 559 C.1. Vectors, Spans, and Bases 559 C.2. Dot Products and Projections 563 C.3. Orthonormal Bases 567 C.4. Matrices 569 Exercises 575 Appendix D. Review of Chapters 1–4 579 D.1. R Infrastructure 579 D.2. Data 580 D.3. Probability Basics 581 D.4. Probability Toolkit 581 viii Contents D.5. Inference 582 D.6. Important Distributions 583 Exercises 583 Hints, Answers, and Solutions to Selected Exercises 587 Bibliography 599 Index to R Functions, Packages, and Data Sets 605 Index 609 Preface Intended Audience As the title suggests, this book is intended as an introduction to both the foun- dations and applications of statistics. It is an introduction in the sense that it does not assume a prior statistics course. But it is not introductory in the sense of being suitable for students who have had nothing more than the usual high schoolmathematicspreparation. Thetargetaudienceisundergraduatestudentsat the equivalent of the junior or senior year at a college or university in the United States. Students should have had courses in differential and integral calculus, but not much more is required in terms of mathematical background. In fact, most of my students have had at least another course or two by the time they take this course, but the only courses that they have all had is the calculus sequence. The majority of my students are not mathematics majors. I have had students from biology, chemistry, computer science, economics, engineering, and psychology, and I have tried to write a book that is interesting, understandable, and useful to students with a wide range of backgrounds and career goals. This book is suitable for what is often a two-semester sequence in “mathe- matical statistics”, but it is different in some important ways from many of the books written for such a course. I was trained as a mathematician first, and the book is clearly mathematical at some points, but the emphasis is on the statistics. Mathematics and computation are brought in where they are useful tools. The result is a book that stretches my students in different directions at different times – sometimes statistically, sometimes mathematically, sometimes computationally. The Approach Used in This Book Features of this book that help distinguish it from other books available for such a course include the following: ix