BE540W Introduction to Probability Page 1 of 54 Unit 2 Introduction to Probability Page Topics 1. Why We Need Probability ………….………………………..……..……… 2 2. Definition Probability Model ……………..………………………….…… 4 3. The “Equally Likely” Setting: Introduction to Probability Calculations ……..……………………..……………….…….. 9 4. Some Useful Basics ……..…………………………..…..………………… 13 a. Sample Space, Elementary Outcomes, Events ………………………… 13 b. “Mutually Exclusive” and “Statistical Independence” Explained ……. 16 c. Complement, Union, and Intersection ………………………………… 18 5. Independence, Dependence, and Conditional Probability ……………… 20 6. Some Useful Tools for Calculating Probabililties ………………………. 25 a. The Addition Rule ……………………………………………………. 25 b. The Multiplication Rule ……………………………………………… 26 a. Theorem of Total Probability …………………………………….…… 27 b. Bayes Rule ……………………………………………………….…… 29 7. A Simple Probability Model: The Bernoulli Distribution …… 31 8. Probability in Diagnostic Testing ………………………..………… 33 a. Prevalence………………………………………………….. 33 b. Incidence ………………………………….…………..….…. 33 c. Sensitivity, Specificity ……………………………………… 34 d. Predictive Value Positive, Negative Test …………………… 37 9. Probability and Measures of Association for the 2x2 Table……….... 39 a. Risk …………………………………………………..…….. 39 b. Odds ………………………………..……………………… 41 c. Relative Risk ………………………………………….…… 43 d. Odds Ratio ………………………………………………...… 45 Appendices 1. Some Elementary Laws of Probability ……………………….…… 48 2. Introduction to the Concept of Expected Value .………….………… 51 BE540W Introduction to Probability Page 2 of 54 1. Why We Need Probability In topic 1 (Summarizing Data), our scope of work was a limited one, describe the data at hand. We learned some ways to summarize. • Histograms, frequency tables, plots; • Means, medians; and • Variance, SD, SE, MAD, MADM. And previously, (Course Introduction), we acknowledged that, in our day to day lives, we have some intuition for probability. • What are the chances of winning the lottery? (simple probability) • From an array of treatment possibilities, each associated with its own costs and prognosis, what is the optimal therapy? (conditional probability) Chance - In these, we were appealing to “chance”. The notion of “chance” is described using concepts of probability and events. We recognize this in such familiar questions as: • What are the chances that a diseased person will obtain a test result that indicates the same? (Sensitivity) • If the experimental treatment has no effect, how likely is the observed discrepancy between the average response of the controls and the average response of the treated? (Clinical Trial) BE540W Introduction to Probability Page 3 of 54 To appreciate the need for probability, consider the conceptual flow of this course: • Initial lense - Describe the data at hand (Topic 1 – Summarizing Data) • Enlarged lense – The data are a sample from a population (Topic 3 – Populations and Samples). Thus, we are relaxing our previous view of the data as a “given”. • Same enlarged lense – If this sample is one of a collection of equally likely samples that could have been obtained, then what is the likelihood of having obtained the particular sample that is ours? (Topic 2 – Introduction to Probability). • Often (and for purposes of this course), the data at hand can be reasonably regarded as a simple random sample from a particular population distribution – eg. - Bernoulli, Binomial, Normal. (Topic 4 – Bernoulli and Binomial Distribution, Topic 5 – Normal Distribution). • Estimation – We seek an understanding, through estimation, of the source population that gave rise to the observed data. Eg – we might want to estimate the value of the mean parameter (µ) of the population distribution that gave rise to our sample (Topic 6 – Estimation and Topic 9-Correlation and Regression). • Hypothesis Testing - Also of interest are some tools for a formal comparison of competing explanations. (Topic 7-Hypothesis Testing, Topic 8 –Chi Square Tests) Now we have a better sense of why we need probability. Of interest are such things as: • If the population source is known, what are the chances of obtaining a particular outcome? A particular collection of outcomes? A particular sample of data? • We will use the tools of probability in confidence interval construction. • We will use the tools of probability in statistical hypothesis testing. BE540W Introduction to Probability Page 4 of 54 2. Definition Probability Model Setting - • The source population is assumed known. Eg - we might conceive of the source population as a big bucket of different colored balls that has been uniformly stirred. • The sample is assumed to be a simple random sample; thus, it is just one of a collection of “equally likely’ samples. Eg – imagine you have reached into the bucket with your hand and scooped out a handful of colored balls. • Note – An introduction to populations and samples is provided in Topic 3. Question - If the available sample is representative of the source population, what are the “chances” of obtaining the observed values? This is a “frequentist” approach to probability. It is not the only approach. Alternative approaches - • Bayesian - “This is a fair coin. It lands “heads” with probability 1/2. • Frequentist – “In 100 tosses, this coin landed heads 48 times”. • Subjective - “This is my lucky coin”. BE540W Introduction to Probability Page 5 of 54 Probabilities and probability distributions are nothing more than extensions of the ideas of relative frequency and histograms, respectively: Ignoring certain mathematical details, a discrete probability distribution consists of two ingredients: 1. The possible values a random value can assume, together with 2. The probabilities with which these values are assumed. Note: An elaboration of this intuition is required for the definition of a continuous probability distribution. Example - • Suppose the universe of all university students is known to include men and women in the ratio 53:47. • Consider the random variable, X = gender of an individual student For convenience, we will say X = 0 when the student is “male” X = 1 when the student if “female” BE540W Introduction to Probability Page 6 of 54 • We have what we need to define a discrete probability distribution: Ingredient 1 - Ingredient 2 - Possible value of X is represented as x Probability [ X = x ] 0 = male 0.53 1 = female 0.47 Be sure to check that this enumeration of Be sure to check that these probabilities add all possible outcomes is “exhaustive”. up to 100% or a total of 1.00. More formally, probability can be defined as • the chance of observing a particular outcome (discrete), or • the likelihood of an event (continuous). • The concept of probability assumes a stochastic or random process: i.e., the outcome is not predetermined – there is an element of chance. • In discussing discrete probabilities, we assign a numerical weight or “chances” to each outcome. This “chance of” an event is its likelihood of occurrence. BE540W Introduction to Probability Page 7 of 54 Notation - The probability of outcome O is denoted P(O) i i • The probability of each outcome is between 0 and 1, inclusive: 0 <= P(O) <= 1 for all i i • Conceptually, we can conceive of a population as a collection of “elementary” events or sample points. (Eg – the population might be the collection of colored balls in the bucket mentioned earlier in these notes) Here, “elementary” is the idea that such an event cannot be broken down further. The probabilities of all possible elementary outcomes sum to 1. ∑ P(O ) =1 (something happens) i all possible elementary outcomes O i • An event E might be one or several elementary outcomes, O. If an event E is certain, then it occurs with probability 1. This allows us to write P(E) = 1. • If an event E is impossible, P(E) = 0. BE540W Introduction to Probability Page 8 of 54 Some More Formal Language 1. (discrete case) A probability model is the set of assumptions used to assign probabilities to each outcome in the sample space. (Eg – in the case of the bucket of colored balls, the assumption might be that the collection balls has been uniformly mixed so that each ball has the same chance of being picked when you scoop out one ball) The sample space is the universe, or collection, of all possible outcomes. Eg – the collection of colored balls in the bucket. 2. A probability distribution defines the relationship between the outcomes and their likelihood of occurrence. 3. To define a probability distribution, we make an assumption (the probability model) and use this to assign likelihoods. Eg – Suppose we imagine that the bucket contains 50 balls, 30 green and 20 orange. Next, imagine that the bucket has been uniformly mixed. If the game of “sampling” is “reach in and grab ONE”, then there is a 30/50 chance of selecting a green ball and a 20/50 chance of selecting an orange ball. 4. When the outcomes are all equally likely, the model is called a uniform probability model In the pages that follow, we will be working with this model and then some (hopefully) straightforward extensions. From there, we’ll move on to probability models for describing the likelihood of a sample of outcomes where the chances of each outcome are not necessarily the same (Bernoulli, Binomial, Normal, etc). BE540W Introduction to Probability Page 9 of 54 3. The “Equally Likely” Setting Introduction to Probability Calculations An “equally likely” setting is the game of rolling a die – There are 6 possible outcomes: {1, 2, 3, 4, 5, 6}. The probability of each is: P(1) = 1/6 P(2) = 1/6 . . . P(6) = 1/6 Sum = 1 Another “equally likely” setting is the tossing of a coin – There are 2 possible outcomes in the set of all possible outcomes {H, T}. Here, “H” stands for “heads” and “T” stands for “tails”. Probability Distribution: O P(O) i i H .5 T .5___ Sum =1 BE540W Introduction to Probability Page 10 of 54 Here is another “equally likely” setting – The set of all possible samples of digits of sample size n that can be taken, with replacement, from a population of size N. E.g., for N=3, n=2: Sample Space: S = { (1,1), (1,2), (1,3), (2,2), (2,1), (2,3), (3,1), (3,2), (3,3) } Probability Model: Assumption: equally likely outcomes, with Nn = 32 = 9 outcomes Probability Distribution: Outcome, O P(O) i i (1,1) 1/9 (1,2) 1/9 … … (3,3) 1/9___ Sum =1 Note – More on the ideas of “with replacement” and “without replacement” later. Another “equally likely” setting – Toss 2 coins Set of all possible outcomes: S = {HH, HT, TH, TT} Probability Distribution: Outcome, O P(O) i i HH ¼ = .25 HT ¼ = .25 TH ¼ = .25 TT ¼ = .25 Sum =1