Probability, outside the classroom David Aldous 4 March 2016 DavidAldous Probability,outsidetheclassroom I have an ongoing project to articulate what 350 years of mathematical probability tells us about the real world. What does theory tell us that is interesting or useful, and demonstrably true via empirical data? A Berkeley undergraduate course. Reviews of non-technical books. A disorganized collection of web pages. (my retirement project) “A map of the world of chance” – a list of the 100 diverse contexts in which we do (or should) perceive chance. This talk – to public audience, or to probabilists to encourage them to think about the connection between math and reality. DavidAldous Probability,outsidetheclassroom Two main points Here’s one page illustrating Probability inside the classroom. [show page] • Can we replace such “made-up stories” with real data? More ambitiously, could we only talk about topics where we have interesting real data? DavidAldous Probability,outsidetheclassroom Contents 1 DiscreteProbabilityDistributions 1 1.1 SimulationofDiscreteProbabilities................... 1 1.2 DiscreteProbabilityDistributions.................... 18 2 ContinuousProbabilityDensities 41 2.1 SimulationofContinuousProbabilities................. 41 2.2 ContinuousDensityFunctions...................... 55 3 Combinatorics 75 3.1 Permutations............................... 75 3.2 Combinations............................... 92 3.3 CardShuffling...............................120 4 ConditionalProbability 133 4.1 DiscreteConditionalProbability ....................133 4.2 ContinuousConditionalProbability...................162 4.3 Paradoxes.................................175 5 DistributionsandDensities 183 5.1 ImportantDistributions.........................183 5.2 ImportantDensities ...........................205 6 ExpectedValueandVariance 225 6.1 ExpectedValue..............................225 6.2 VarianceofDiscreteRandomVariables.................257 6.3 ContinuousRandomVariables......................268 7 SumsofRandomVariables 285 7.1 SumsofDiscreteRandomVariables ..................285 7.2 SumsofContinuousRandomVariables.................291 8 LawofLargeNumbers 305 8.1 DiscreteRandomVariables .......................305 8.2 ContinuousRandomVariables......................316 v • Instead of organizing by methodology, I want to illustrate the diversity of contexts in which we perceive chance – the “map of the world of chance”. DavidAldous Probability,outsidetheclassroom I can’t find any serious academic attempt to do this; though Nate Silver’s The Signal and the Noise chapters mostly correspond closely to contexts on the list. • (3) accuracy of opinion polls vs expert assessments. • (6) sports betting. • (7) baseball player’s performance. • (8) professional poker. • (16) flu pandemics. • (26) weather. • (32) mortgage default likelihoods. • (34) predicting business cycle/economic indicators. • (36) terrorism. • (45) climate change. • (65) predicting earthquakes. • (81) stock market, efficient market hypothesis, bubbles. • (84) Herding, overconfidence. DavidAldous Probability,outsidetheclassroom 18 lectures in my course • Everyday perception of chance • Ranking and rating • Risk to individuals: perception and reality • Luck • A glimpse at probability research: spatial networks on random points • Prediction markets, fair games and martingales • Science fiction meets science • Coincidences, near misses and one-in-a-million chances. • Psychology of probability: predictable irrationality • Mixing: physical randomness, the local uniformity principle and card shuffling • Game theory •TheKellycriterionforfavorablegames: stockmarketinvestingforindividuals • Toy models in population genetics: some mathematical aspects of evolution • Size-biasing, regression effect and dust-to-dust phenomena • Toy models of human interaction: use and abuse • Short/Medium term predictions in politics and economics • Tipping points and phase transitions • Coding and entropy DavidAldous Probability,outsidetheclassroom My ideal is to build each lecture around some “anchor data”, and to make it interesting enough to be re-used in this public talk. This is hard to do! In this talk I’ll touch upon 7 of the lecture topics – initially very briefly and then at greater length. Lecture Topic 1. Here’s the anchor data for my “ranking and rating” lecture. [show Elo soccer rankings] [chat] DavidAldous Probability,outsidetheclassroom Lecture Topic 2. Nobel Prize winner Kahneman’s wonderful book Thinking, fast and slow deals with various cognitive biases, many involving probability and risk. I cannot supply “value added” so I do one class just using the book material. In particular there are fun “psychology” experiments you can try on your friends. [explain red-black bets] However that research is mostly based on answers to hypothetical questions involving probability and risk. I am interested in a different question: Lecture Topic 3. In what contexts of everyday life do we think in terms of chance? [show my Bing page] DavidAldous Probability,outsidetheclassroom Pedagogic point: where you get data matters! Our data on references to chance on tweets is quite different. This fits my theme of replacing “made up stories” with real data, for making a map/list of actual contexts where chance arises. [show Rescher] DavidAldous Probability,outsidetheclassroom Lecture Topic 4. The Kelly criterion. This is my “science cocktail party” conversation. Here is my anchor data. [show] My math discussion below is very oversimplified but makes the main points. Note that almost all of “mathematical finance” concerns short-term speculative trading and is irrelevant to you as an individual investor. I will talk about long-term investment from an individual’s viewpoint. DavidAldous Probability,outsidetheclassroom