Introduction to Machine Learning CMU-10701 2. Basic Statistics Barnabás Póczos & Alex Smola Remember the color coding Important Not so important You can sleep now… 2 Please ask Questions and give us ! Feedbacks 3 2. Basic Statistics Essential tools for data analysis 4 Outline Theory: • Probabilities: – Probability measures, events, random variables, conditional probabilities, dependence, expectations, etc • Bayes rule • Parameter estimation: – Maximum Likelihood Estimation (MLE) – Maximum a Posteriori (MAP) Application: Naive Bayes Classifier for • Spam filtering • “Mind reading” = fMRI data processing 5 What is the probability? Probabilities Bayes Kolmogorov 6 Probability σ • Sample space, Events, -Algebras • Axioms of probability, probability measures – What defines a reasonable theory of uncertainty? •Random variables: – discrete, continuous random variables • Joint probability distribution • Conditional probabilities • Expectations • Independence, Conditional independence 7 Sample space Ω Def: A sample space is the set of all possible outcomes of a (conceptual or physical) random Ω experiment. ( can be finite or infinite.) Examples: −Ω may be the set of all possible outcomes of a dice roll (1,2,3,4,5,6) -Pages of a book opened randomly. (1-157) -Real numbers for temperature, location, time, etc 8 Events We will ask the question: What is the probability of a particular event? Ω Def: Event A is a subset of the sample space Examples: What is the probability of − the book is open at an odd number − rolling a dice the number <4 − a random person’s height X : a<X<b 9 Probability Def: Probability P(A), the probability that event (subset) A happens, is a function that maps the event A onto the interval [0, 1]. P(A) is also called the probability measure of A. outcomes in which A is false Ω sample space 1,3,5,6 outcomes in which A is true 2,4 Example: What is the probability that the number on the dice is 2 or 4? P(A) is the volume of the area. 10