To HARRIET A.M.M. To my GRANDCHILDREN F.A.G. To JOAN, LISA, and KARIN D .. C.B. Library of Congress Cataloging in Publication Data Mood, Alexander McFar1ane, 1913- Introduction to the theory of statistics. (McGraw-Hi1l series in probability and statistics) Bibliography: p. 1. Mathematical statistics. I. Graybill. Frank1in A., joint author. II. .Boes, Duane C., joint author. III. Title. QA276.M67 1974 519.5 73-292 ISBN 0-{)7-042864-6 INTRODUCTION TO THE THEORY OF STATISTICS Copyright © 1963. t 974 by McGraw-Hill, Inc. All rights reserved. Copyright 1950 by McGraw-Hili, Inc. All rights reserved. Printed in the United States of America. No part of this pubJication may be reproduced, stored in a retrieval system, or transmitted, in any form or by any means, electronic. mechanical, photocopying, recording, or otherwise, without the prior written permission of the publisher. 6789 10 KPKP 7832109 This book was set in Times Roman. The editors were Brete C. Harrison and Madelaine Eichberg; the cover was designed by Nicholas Krenitsky; and the production supervisor was Ted Agrillo. The drawings were done by Oxford Il1ustrators Limited. The printer and binder was Kinsport Press, Inc. CONTENTS ... Preface to the Third Edition Xlll Excerpts from the First and Second Edition Prefaces xv I Probability 1 1 Introduction and Summary 1 2 Kinds of Probability 2 2.1 Introduction 2 2.2 Classical or a Priori Probability 3 2.3 A Posteriori or Frequency Probability 5 3 Probability-Axiomatic 8 3.1 Probability Models 8 3.2 An Aside-Set Theory 9 3.3 Definitions of Sample Space and Event 14 3.4 Definition of Probability 19 3.5 Finite Sample Spaces 25 3.6 Conditional Probability and Independence 32 vi CONTENTS II Random Variables, Distribution Functions, and Expectation 51 1 Introduction and Summary 51 2 Random Variable and Cumulative Distribution Function 52 2.1 Introduction 52 2.2 Definitions 53 3 Density Functions 57 3.1 Discrete Random Variables 57 3.2 Continuous Random Variables 60 3.3 Other Random Variables 62 4 Expectations and Moments 64 4.1 Mean 64 4.2 Variance 67 .4.3 Expected Value of a Function of a Random Variable 69 4.4 Inequali ty 71 Ch~byshev 4.5 Jensen Inequality 72 4.6 Moments and Moment Generating Functions 72 ill Special Parametric Families of Univariate Distributions 85 1 Introduction and Summary 85 2 Discrete Distributions 86 2.1 Discrete Uniform Distribution 86 2.2 Bernoulli and Binomial Distributions 87 2.3 Hypergeometric Distribution 91 2.4 Poisson Distribution 93 2.5 Geometric and Negative Binomial Distributions 99 2.6 Other Discrete Distributions 103 3 Continuous Distributions 105 ).( Uniform or Rectangular Distribution 105 # . (3.2 ) Normal Distribution 107 '-- 3.3 Exponential and Gamma Distributions 111 3.4 Beta Distribution 115 3.5 Other Continuous Distributions 116 4 Comments 119 4.1 Approximations 119 4.2 Poisson and Exponential Relationship 121 4.3 Contagious Distributions and Truncated Distributions 122 CONTENTS vii IV Joint and Conditional Distributions, Stochastic Independence, More Expectation 129 1 Introduction and Summary 129 2 Joint Distribution Functions 130 2.1 Cumulative Distribution Function 130 2.2 Joint Density Functions for Discrete Random Variables 133 2.3 Joint Density Functions for Continuous Random Variables 138 3 Conditional Distributions and Stochastic Independence 143 3.1 Conditional Distribution Functions for Discrete Random Variables 143 3.2 Conditional Distribution Functions for Continuous Random Variables 146 3.3 More on Conditional Distribution Functions 148 3.4 Independence 150 4 Expectation 153 4.1 Definition 153 4.2 Covariance and Correlation Coefficient 155 4.3 Conditional Expectations 157 4.4 Joint Moment Generating Function and Moments 159 4.5 Independence and Expectation 160 4.6 Cauchy-Schwarz Inequality 162 5 Bivariate Normal Distribution 162 5.1 Density Function 162 5.2 Moment Generating Function and Moments 164 5.3 - Marginal and Conditional Densities 167 V Distributions of Functions of Random Variables 175 1 Introduction and Summary 175 2 Expectations of Functions of Random Variables 176 2.1 Expectation Two Ways 176 2.2 Sums of Random Variables 178 2.3 Product and Quotient 180 3 Cumulative-distribution-function Technique 181 3.1 Description of Technique 181 3.2 Distribution of Minimum and Maximum 182 3.3 Distribution of Sum and Difference of Two Random Variables 185 3.4 Distribution of Product and Quotient 187 viii CONTENTS 4 Moment-generating-function Technique 189 4.1 Description of Technique 189 4.2 Distribution of Sums of Independent Random Variables 192 5 The Transformation Y = g(X} 198 5.1 Distribution of Y g(X) 198 5.2 Probability Integral Transform 202 6 Transformations 203 6.1 Discrete Random Variables 203 6.2 Continuous Random Variables 204 VI Sampling and Sampling Distributions 219 1 Introduction and Summary 219 2 Sampling 220 2.1 Inductive Inference 220 2.2 Populations and Samples 222 2.3 Distribution of Sample 224 2.4 Statistic and Sample Moments 226 3 Sample Mean 230 ~Mean and Variance 231 3.2 Law of Large Numbers 231 .3 Central-limit Theorem 233 3.4 Bernoulli and Poisson Distributions 236 3.5 Exponential Distribution 237 3.6 Uniform Distribution 238 3.7 Cauchy Distribution 238 4 Sampling from the Normal Distributions 239 4.1 Role of the Normal Distribution in Statistics 239 4.2 Sample Mean 240 4.3 The Chi-square Distribution 241 4.4 The F Distribution 246 4.5 Student's t Distribution 249 5 Order Statistics ~ 5.1 Definition and Distributions 5.2 Distribution of Functions of Order Statistics 254 5.3 Asymptotic Distributions 256 5.4 Sample Cumulative Distribution Function 264 CONTENTS ix VII Parametric Point Estimation 271 1 Introduction and Summary 271 2 Methods of Finding Estimators 273 2.1 Methods of Moments 274 2.2 Maximum Likelihood 276 2.3 Other Methods 286 3 Properties of Point Estimators 288 3.1 Closeness 288 3.2 Mean-squared Error 291 3.3 Consistency and BAN 294 3.4 Loss and Risk Functions 297 4 Sufficiency 299 4.1 Sufficient Statistics 300 4.2 Factorization Criterion 307 4.3 Minimal Sufficient Statistics 311 4.4 Exponential Family 312 5 Unbiased Estimation 315 5.1 Lower Bound for Variance 315 5.2 Sufficiency and Completeness 321 6 Location or Scale Invariance 331 6.1 Location Invariance 332 6.2 Scale Invariance 336 7 Bayes Estimators 339 7.1 Posterior Distribution 340 7.2 Loss-function Approach 343 7.3 Minimax Estimator 350 8 Vector of Parameters 351 9 Optimum Properties of Maximum-likelihood Estimation 358 vm Parametric Interval Estimation 372 1 Introduction and Summary 372 2 Confidence Intervals 373 2.1 An Introduction to Confidence Intervals 373 2.2 Definition of Confidence Interval 377 2.3 Pivotal Quantity 379 X CONTENTS 3 Sampling from the Normal Distribution 381 3.1 Confidence Interval for the Mean 381 3.2 Confidence Interval for the Variance 382 3.3 Simultaneous Confidence Region for the Mean and Variance 384 3.4 Confidence Interval for Difference in Means 386 4 Methods of Finding Confidence Intervals 387 4.1 Pivotal-quantity Method 387. 4.2 Statistical Method 389 5 Large-sample Confidence Intervals 393 6 Bayesian Interval Estimates 396 IX Tests of Hypotheses 401 1 Introduction and Summary 401 2 Simple Hypothesis versus Simple Alternative 409 2.1 Introduction 409 2.2 Most Powerful Test 410 2.3 Loss Function 414 3 Composite Hypotheses 418 3.1 Generalized Likelihood-ratio Test 419 3.2 Uniformly Most Powerful Tests 421 3.3 Unbiased Tests 425 3.4 Methods of Finding Tests 425 4 Tests of Hypotheses-Sampling from the Normal Distribution 428 4.1 Tests on the Mean 428 4.2 Tests on the Variance 431 4.3 Tests on Several Means 432 4.4 Tests on Several Variances 438 5 Chi-square Tests. 440 5.1 Asymptotic Distribution of Generalized Likelihood-ratio 440 5.2 Chi-square Goodness··of-fit Test 442 5.3 Test of the Equality of Two Multinomial Distributions and Generalizations 448 5.4 Tests of Independence in Contingency Tables 452 6 Tests of Hypotheses and Confidence Intervals 461 7 Sequenti'al Tests of Hypotheses 464 7.1 Introduction 464 CONTENTS xi 7.2 Definition of Sequential Probability Ratio Test 466 7.3 Approximate Sequential Probability Ratio Test 468 7.4 Approximate Expected Sample Size of Sequential Probability Ratio Test 470 X Linear Models 482 1 Introduction and Summary 482 2 Examples of the Linear Model 483 3 Definition of Linear Model 484 4 Point Estimation~Case A 487 5 Confidence Intervals-Case A 491 6 Tests of Hypotheses-Case A 494 7 Point Estimation-Case B 498 XI N onparametric Methods 504 1 Introduction and Summary 504 2 Inferences Concerning a Cumulative Distribution Function 506 2.1 Sample or Empirical Cumulative Distribution Function 506 2.2 Kolmogorov-Smirnov Goodness-of-fit Test 508 2.3 Confidence Bands for Cumulative Distribution Function 511 3 Inferences Concerning Quantiles 512 3.1 Point and Interval Estimates of a Quantile 512 3.2 Tests of Hypotheses Concerning Quantiles 514 4 Tolerance Limits 515 5 Equality of Two Distributions 518 5.1 Introduction 518 5.2 Two-sample Sign Test 519 5.3 Run Test 519 5.4 Median Test 521 5.5 Rank-sum Test 522 Appendix A. Mathematical Addendum 527 1 Introduction 527 xii CONTENTS 2 Noncalculus 527 2.1 Summation and Product Notation 527 2.2 Factorial and Combinatorial Symbols and Conventions 528 2.3 Stirling's Formula 530 2.4 The Binomial and Multinomial Theorems 530 3 Calculus 531 3.1 Preliminaries 531 3.2 Taylor Series 533 3.3 The Gamma and Beta Functions 534 Appendix B. Tabular Summary of Parametric Families of Distributions 537 1 Introduction 537 Table 1. Discrete Distributions 538 Table 2. Continuous Distributions 540 Appendix C. References and Related Reading 544 Mathematics Books 544 Probability Books 544 Probability and Statistics Books 545 Advanced (more advanced than MGB) 545 Intermediate (about the same level as MGB) 545 Elementary (less advanced than MGB, but calculus prerequisite) 546 Special Books 546 Papers 546 Books of Tables 547 Appendix D. Tables 548 1 Description of Tables 548 Table 1. Ordinates of the Normal Density Function 548 Table 2. Cumulative Normal Distribution 548 Table 3. Cumulative Chi-square Distribution 549 Table 4. Cumulative F Distribution 549 Table 5. Cumulative Student's t Distribution 550 Index 557 PREFACE TO THE THIRD EDITION The purpose of the third edition of this book is to give a sound and self-con tained (in the sense that the necessary probability theory is included) introduction to classical or mainstream statistical theory. It is not a statistical-methods cookbook, nor a compendium of statistical theories, nor is it a mathematics book. The book is intended to be a textbook, aimed for use in the traditional full-year upper-division undergraduate course in probability and statistics, or for use as a text in a course designed for first-year graduate students. The latter course is often a "service course," offered to a variety of disciplines. No previous course in probability or statistics is needed in order to study the book. The mathematical preparation required is the conventional full-year calculus course which includes series expansion, mUltiple integration, and par tial differentiation. Linear algebra is not required. An attempt has been made to talk to the reader. Also, we have retained the approach of presenting the theory with some connection to practical problems. The book is not mathe matically rigorous. Proofs, and even exact statements of results, are often not given. Instead, we have tried to impart a "feel" for the theory. The book is designed to be used in either the quarter system or the semester system. In a quarter system, Chaps. I through V could be covered in the first