Robert Jonsson Exercises in Statistical Inference with detailed solutions 2 Exercises in Statistical Inference with detailed solutions 1st edition © 2014 Robert Jonsson ISBN: 978-87-403-0752-8 87-403-0752-2 3 Exercises in Statistical Inference with detailed solutions Contents Contents About the author 7 1 Introduction 8 1.1 Purpose of this book 8 1.2 Chapter content and plan of the book 8 1.3 Statistical tables and facilities 10 2 Basic probability and mathematics 12 2.1 Probability distributions of discrete and continuous random variables 12 2.2 Some distributions 17 2.3 Mathematics 27 2.4 Final words 33 4 Exercises in Statistical Inference with detailed solutions Sampling Distributions 34 3.1 Some exact sampling distributions 35 3.2 Sample moments 37 3.3 Asymptotic and approximate results in sampling theory 39 3.4 Final words 44 4 Point estimation 46 4.1 Concepts 46 4.2 Requirements on estimators 49 4.3 Estimation methods 51 4.4 Final words 61 5 Interval estimation 64 5.1 Concepts 64 5.2 CIs in small samples by means of pivotal statistics 65 5.3 Approximate CIs in large samples based on Central Limit Theorems 71 5.4 Some further topics 74 5.5 Final words 79 Exercises in Statistical Inference with detailed solutions Contents 6 Hypothesis Testing 82 6.1 Concepts 82 6.2 Methods of finding tests 86 6.3 The power of normally distributed statistics 120 6.4 Adjusted p-values for simultaneous inference 125 6.5 Randomized tests 127 6.6 Some tests for linear models 128 6.7 Final wjords 144 Answers to Supplementary Exercises 158 References 197 II jjooiinneedd MMIITTAASS bbeeccaauussee ��ee GGrraadduuaattee PPrrooggrraammmmee ffoorr EEnnggiinneeeerrss aanndd GGeeoosscciieennttiissttss II wwaanntteedd rreeaall rreessppoonnssiibbiillii�� MMaaeerrsskk..ccoomm//MMiittaass �e Graduate Programme for Engineers and Geoscientists Maersk.com/Mitas MMoonntthh 1166 IIII wwwwaaaassss aaaa ccoonnssttrruuccttiioonn Month 16 ssuuppeerrvviiIIss wwoorraa iissnn aa construction tthhee NNoorrtthh SSeeaa supervisor in aaddvviissiinngg aanndd the North Sea hhhheeeellppiinngg ffoorreemmeenn advising and RReeaall wwoorrkk IIIInnnntttteeeerrrrnnnnaaaattttiiiioooonnnnaaaaaallll ooppppoorrttuunniittiieess ssssoollvvee pprroobblleemmhhsseelping foremen ����rrrreeeeeeee wwwwoooooorrrrkk ppllaacceemmeennttss Real work IInntteerrnnaattiioonnaaall opportunities ssolve problems ��rreeee wwooorrk placements 6 Exercises in Statistical Inference with detailed solutions About the author About the author Robert Jonsson got his Ph.D. in Statistics from the Univ. of Gothenburg, Sweden, in 1983. He has been doing research as well as teaching undergraduate and graduate students at Dept. of Statistics (Gothenburg), Nordic School of Public Health (Gothenburg) and Swedish School of Economics (Helsinki, Finland). His researches cover theoretical statistics, medical statistics and econometrics that in turn have given rise to 14 articles in refereed international journals and some dozens of national papers. Teaching experience reaches from basic statistical courses for undergraduates to Ph.D. courses in Statistical Inference, Probability and Stochastic processes. 7 Exercises in Statistical Inference with detailed solutions Introduction 1 Introduction 1.1 Purpose of this book The book is designed for students in statistics at the master level. It focuses on problem solving in the field of statistical inference and should be regarded as a complement to text books such as Wackerly et al 2007, Mathematical Statistics with Applications or Casella & Berger 1990, Statistical Inference. The author has noticed that many students, although being well aware of the statistical ideas, fall short when being faced with the task of solving problems. This requires knowledge about statistical theory, but also about how to apply proper methodology and useful tricks. It is the aim of the book to bridge the gap between theoretical knowledge and problem solving. Each of the following chapters contains a minimum of the theory needed to solve the problems in the Exercises. The latter are of two types. Some exercises with solutions are interspersed in the text while others, called Supplementary Exercises, follow at the end of the chapter. The solutions of the latter are found at the end of the book. The intention is that the reader shall try to solve these problems while having the solutions of the preceding exercises in mind. Towards the end of the following chapters there is a section called ‘Final Words’. Here some important aspects are considered, some of which might have been overlooked by the reader. 1.2 Chapter content and plan of the book Emphasis will be on the kernel areas of statistical inference: Point estimation – Confidence Intervals – Test of hypothesis. More specialized topics such as Prediction, Sample Survey, Experimental Design, Analysis of Variance and Multivariate Analysis will not be considered since they require too much space to be accommodated here. Results in the kernel areas are based on probability theory. Therefore we first consider some probabilistic results, together with useful mathematics. The set-up of the following chapters is as follows. • Ch. 2 Basic properties of discrete and continuous (random) variables are considered and examples of some common probability distributions are given. Elementary pieces of mathematics are presented, such as rules for derivation and integration. Students who feel that their prerequisites are insufficient in these topics are encouraged to practice hard, while others may skip much of the content of this chapter. • Ch. 3 The chapter is mainly devoted to sampling distributions, i.e. the distribution of quantities that are computed from a sample such as sums and variances. In more complicated cases methods are presented for obtaining asymptotic or approximate formulas. Results from this chapter are essential for the understanding of results that are derived in the subsequent chapters. 8 Exercises in Statistical Inference with detailed solutions Introduction • Ch. 4 Important concepts in point estimation are introduced, such as likelihood of a sample and sufficient statistics. Statistics used for point estimation of unknown quantities in the population are called estimators. (Numerical values of the latter are called estimates.) Some requirements on ‘good’ estimators are mentioned, such as being unbiased, consistent and having small variance. Four general methods for obtaining estimators are presented: Ordinary Least Squares (OLS), Moment, Best Linear Unbiased Estimator (BLUE) and Maximum Likelihood (ML). The performance of various estimators is compared. Due to limited space other estimation methods have to be omitted. • Ch. 5 The construction of confidence intervals (CIs) for unknown parameters in the population by means of so called pivotal statistics is explained. Guide lines are given for determining the sample size needed to get a CI of certain coverage probability and of certain length. It is also shown how CIs for functions of parameters, such as probabilities, can be constructed. • Ch. 6 Two alternative ways of testing hypotheses are described, the p-value approach and the rejection region (RR) approach. When a statistic is used for testing hypotheses it is called a test statistic. Two general principles for constructing test statistics are presented, the Chi-square principle and the Likelihood Ratio principle. Each of these gives raise to a large number of well-known tests. It’s therefore a sign of statistical illiteracy when referring to a test as the Chi- Square test (probably supposed to mean the well-known test of independency between two qualitative variables). Furthermore, some miscellaneous methods are presented. A part of the chapter is devoted to nonparametric methods for testing goodness-of-fit, equality of two or more distributions and Fisher’s exact test for independency. A general expression for the power (ability of a test to discriminate between the alternatives) is derived for (asymptotically) normally distributed test statistics and is applied to some special cases. When several hypotheses are tested simultaneously, we increase the probability of rejecting a hypothesis when it in fact is true. (This is one way to ‘lie’ when using statistical inference, more examples are given in the book.) One solution of this problem, called the Bonferroni-Holm correction is presented. We finally give some tests for linear models, although this topic perhaps should require their own book. Here we consider the classical Gauss-Markov model and simple cases of models with random coefficients. 9 Exercises in Statistical Inference with detailed solutions Introduction From the above one might get the impression that statistical testing is a more ‘important’ in some sense than point and interval estimation. This is however not the case. It has been noticed that good point estimators also work well for constructing good CIs and good tests. (See e.g. Stuart et al 1999, p. 276.) A frequent question from students is: Which is best, to make a CI or to make a test? A nice answer to this somewhat controversial question can be found in an article by T. Wonnacott, 1987. He argues that in general a CI is to be preferred in front of a test because a CI is more informative. For the same reason he argues for a p-value approach in front of a RR approach. However, in practice there are situations where the construction of CIs becomes too complicated. Also the computation of p-values may be complicated. E.g. in nonparametric inference (Ch. 6.2.4) it is often much easier to make a test based on the RR approach than to use the p-value approach. The latter in turn being simpler than making a CI. An approach based on testing is also much easier to use when several parameters have to be estimated simultaneously. 1.3 Statistical tables and facilities A great deal of the problem solving is devoted to computation of probabilities. For continuous variables this means that areas under frequency curves have to be computed. To this end various statistical tables are available. When using these there are two different quantities of interest. - Given a value on the x-axis, what is the probability of a larger value, i.e. how large is the area under the curve above the value on the x-axis? This may be called computation of a p-value. - Given a probability, i.e. an area under curve, what is the value on the x-axis that produced the probability? This may be called computation of an inverse p-value. Statistical tables can show lower-tail areas or upper-tail areas. Lower-tail areas are areas below values on the x-axis and upper-tail areas are areas above. The reader should watch out carefully whether it is required to search for a p-value or an inverse p-value and whether the table show lower-or upper-tail areas. This seems to actually be a stumbling block for many students. It may therefore be helpful to remember some special cases for the normal-, Student’s T-, Chi-square- and F-distributions. (These will be defined in Ch. 2.2.2 and Ch. 3.1.) The following will serve as hang-ups: - In the normal distribution the area under curve above 1.96 is 0.025. The area under curve below 1.96 is thus 1-0.025=0.975. - In Student’s T distribution one needs to know the degrees of freedom (df) in order to determine the areas. With df = 1 the area under curve above 12.706 is 0.025. - In the Chi-square distribution with df = 1 the area under curve above e3.84(cid:124)(1.96)2 is 2(cid:152)0.025(cid:32)0.05. - In the F distribution one needs to know a pair of degrees of freedoms sometimes denoted (numerator,denominator)(cid:32)(f1, f2). With f1 =1= f2the area under curve above 161.45 (cid:124)(12.706)2is 0.025. 10