Probability

1
Probability Certainty
2
Overview

• History of probability theory
• Some basic probability theory
• Calculating simple probabilities
• Combining mutually-exclusive probabilities
• Combining independent probabilities
• More complex probabilities
• Calculating conditional probabilities
• Bayes Theorem, and why we should care about it
• A test case The Lets Make A Deal! problem

3
History of probability theory

• Compress all of human history (350K years) in one
  24-hour day
• the first recorded general problem representation
  (geometry, invented by Thales of Miletus about
  450 B.C.) would have appeared only 9 minutes and
  30 seconds ago
• the first systematic large-scale collection of
  empirical facts (Tycho Brahes collection of
  astronomical observations) would have appeared
  only a minute and a half ago
• the first mathematical equation which was able to
  predict an empirical phenomena (Newtons 1697
  equation for planetary motion) would have
  appeared only one minute and twelve seconds ago
• Probability theory appeared between 1654 (a
  minute and a half ago) and 1843 (34 seconds ago).

4
History of probability theory

• The emergence of elementary probability theory in
  the 1650s met with enormous resistance and lack
  of comprehension when it was first introduced,
  despite its formal character, its utility, and
  (what we now recognize as) its simplicity.
• The difficult points (Margolis, 1993) were
  philosophical rather than mathematical the
  notions that one could count possibilities that
  had never actually existed, and that order could
  be obtained from randomness.

5
Basic probability theory Example 1

• A boring standard example

How likely is it that we will throw a 6 with one
dice?

• Basic principle The probability of any
  particular event is equal to the ratio of the
  number of ways the event can happen over the
  number of ways anything can happen ( the number
  of ways the event can fail to happen the number
  of ways it can happen).

6
Basic probability theory Example 2
How likely is it that we will throw a 7 with two
die?

• There is more than one way for the event to
  happen 6 1, 16, 5 2, 25, 3 4, 4 3 6
  ways
• There are 36 (6 x 6) ways for anything to happen
• So the probability of 7 with two die is 6/36 or
  1/6.

7
Basic probability theory Example 3
We roll the die 4 times, and never get a seven.
What is the probability that well get on the
5th roll?

• Independent events are events that dont effect
  each others probability. Since the every roll is
  independent of every other, the odds are still
  1/6.

8
Basic probability theory Example 4
We roll the die twice and get a seven both times.
What is the probability of that?

• To combine probabilities of independent events,
  multiply the odds of each event. The odds of each
  7 are 1/6, so the odds of two rolls of 7 in a row
  are 1/6 x 1/6 1/36.

9
Basic probability theory Example 5
We roll the die. What are the odds are getting
either a 7 or a 2?

• Now the events are mutually exclusive if one
  happens, the other cannot. To combine
  probabilities of mutually exclusive events, add
  them together.
• 6/36 odds of 7 1/36 odds of 2 7/36

10
Basic probability theory Example 6.1
We roll the die twice. What are the odds that we
get at least one 7 from the two rolls?

• Can we just add the probabilities of getting a 7
  on each roll?

• No because the events are not mutually
  exclusive anymore
• if we could, wed be above 100- guaranteed a 7-
  after 6 rolls! Where could the guarantee come
  from?

11
Basic probability theory Example 6.2
We roll the die twice. What are the odds that we
get at least one 7 from the two rolls?

• Can we just multiply the probabilities of
  getting a 7 on each roll?

• No because the events are not independent (Why
  not?)
• And 1/6 x 1/6 is less than 1/6- so wed have
  smaller chance with two rolls than with one!

12
Basic probability theory Example 6.3
We roll the die twice. What are the odds that we
get at least one 7 from the two rolls?

• We can turn part of the problem into a problem
  of mutual exclusivity by asking what are the
  odds of there being exactly one seven out of two
  rolls?

• one way is to roll 7 first, but not second
• - the odds of this are 1/6 5/6 (independent
  events) 0.138
• - the odds of rolling 7 second are 5/6 1/6
  (independent events) 0.138
• - since these two outcomes are mutually
  exclusive, we can add them to get 0.138 0.138
  0.277

13
Basic probability theory Example 6.4
We roll the die twice. What are the odds that we
get at least one 7 from the two rolls?

• Now we need to know what are the odds of there
  being two sevens out of two rolls?

• We already know its 1/36 0.03
• So the odds that we get at least one 7 is the
  odds of two 7s the odds of one 7 (mutually
  exclusive events) 0.03 0.27 0.3

14
Basic probability theory The generalization

• Does anyone know how to generalize this
  calculation, so we can easily calculate the odds
  of an event of probability p happening r times
  out of n tries, for any values of p,r, and n?

• The binomial theorem Tune in for the exciting
  next class!

15
Basic probability theory Conditional probability

• What if a dice is biased so that it rolls 6
  twice as often as every other number? How can we
  deal with uneven base rates?
• Why should we care?
• Because real life uses biased dice
• eg. the conditional probability of being
  schizophrenic, given that a person has an
  appointment with a doctor who specializes in
  schizophrenia, is quite different from the
  unconditional probability that a person has
  schizophrenia (the base rate)

16
Basic probability theory Conditional probability

• A particular disorder has a base rate occurrence
  of 1/1000 people. A test to detect this disease
  has a false positive rate of 5. Assume that the
  test diagnoses correctly every person who has the
  disease. What is the chance that a randomly
  selected person with a positive result actually
  has the disease? Take a guess.

Harvard Med School estimates About half said
95. Average response was 56. Only 16 gave the
correct answer.
17
Basic probability theory Conditional probability

• A particular disorder has a base rate occurrence
  of 1/1000 people. A test to detect this disease
  has a false positive rate of 5. Assume that the
  test diagnoses correctly every person who has the
  disease. What is the chance that a randomly
  selected person with a positive result actually
  has the disease?

• Conditional probability P(AB) P(A,B) / P(B)
  P(A and B)/P(B)
• Let A Has the disorder and B Has a positive
  test result
• In 10,000 people, P(A and B)
• P(B)
• Chance that a randomly selected person with a
  positive result actually has the disease

0.001 1000 10
0.05 1000 (false positives) 10 (true
positives) 510
10/510 0.02 or just a 2 chance
18
A generalization Bayes theorem
P(AB) P(BA) P(A) / P(B) Proof By
definition, (1.) P(AB) P(A,B) / P(B) (2.)
P(BA) P(A,B) / P(A) (3.) P(BA) P(A) P(A,B)
Multiply (2.) by P(A) (4.) P(AB) P(B) P(BA)
P(A) Substitute (1.) in (3.) (5.) P(AB)
P(BA) P(A) / P(B)
19
Lets try Bayes theorem
P(AB) P(BA) P(A) / P(B) Let P(A)
Probability of disease 0.001 P(B)
Probability of positive test 0.05 0.001
0.0501 P(BA) Probability of positive
test given disease 1 Then P(AB) P(BA)
P(A) / P(B) (1 x
0.001) / (0.0501) 0.02
20
The Notorious 3-Curtain (Lets Make A Deal)
Problem!

• Three curtains hide prizes. One is good. Two are
  not.
• You choose a curtain.
• The MC opens another curtain. Its not good.
• He gives you the chance to stay with our first
  choice, or switch to the remaining unopened
  curtain.
• Should you stay or switch, or does it matter?

21

• P(AB) P(BA) P(A) / P(B)
• Assume you choose A and switch to the unopened
  one
• LetP(A) Probability it is behind the unopened
  one (B or C) 2/3
• P(B) Probability it is not behind A 2/3
• P(BA) Probability it is not behind A, given
  that it is behind the unopened one (B or C) 1
• Then P(AB) Probability it is behind the
  unopened one (B or C), given that it is not
  behind A
• P(BA) P(A) / P(B)
• (1 x 2/3) / 2/3 1 100 of
  the time.
• But it is given that it is not behind A 2/3 of
  the time, so 2/3 of the time we can be certain of
  winning if we switch!