Probability

1
Probability Certainty
2
Overview

• History of probability theory
• Everything you need to know about probability on
  one slide!
• Some basic probability theory
• Calculating simple probabilities
• Combining mutually-exclusive probabilities
• Combining independent probabilities
• More complex probabilities
• Calculating conditional probabilities
• Bayes' rule, and why we should care about it
• A devious test case The notorious Lets Make A
  Deal! problem

3
Review

• Last time we made three main points
• Information is related to elimination of
  redundancy in any dataset
• Information is related to purpose or goals
• Information in the real world is always uncertain
  or probabilistic

4
Who should you care?

• Probability theory comes into play in four main
  ways in this course
• i.) As the explanation behind distributional
  regularities (normalcy) upon which we will rely
  very heavily to build some useful statistical
  tools
• ii.) In understanding certain systematic errors
  people make in reasoning about diagnosis
• iii.) In understanding how base rates of a
  disease or state can impact on our diagnosis of
  that disease or state
• iv.) In understanding how to decide where to put
  cut-off points for diagnosing a person as
  belonging to a specific diagnostic category
• Generally, probability theory underlies much of
  the reasoning in psychometrics

5
History of probability theory

• Compress all of human history (350K years) in one
  24-hour day
• The first recorded general problem representation
  (geometry, invented by Thales of Miletus about
  450 B.C.) would have appeared only 9 minutes and
  30 seconds ago
• The first systematic large-scale collection of
  empirical facts (Tycho Brahes collection of
  astronomical observations) would have appeared
  only a minute and a half ago
• The first mathematical equation which was able to
  predict an empirical phenomena (Newtons 1697
  equation for planetary motion) would have
  appeared only one minute and twelve seconds ago
• Probability theory appeared between 1654 (a
  minute and a half ago) and 1843 (34 seconds ago).

6
History of probability theory

• The emergence of elementary probability theory in
  the 1650s met with enormous resistance and lack
  of comprehension when it was first introduced,
  despite its formal character, its utility, and
  (what we now recognize as) its simplicity.
• The difficult points were philosophical rather
  than mathematical (Margolis, H. (1993). Paradigms
  and barriers How habits of mind govern
  scientific beliefs. Chicago University Press.)

7
History of probability theory

• In particular, even the greatest geniuses of the
  time had difficulty wrapping their minds around
  two notions
• i.) That one could (and, indeed, had to) count
  possibilities that had never actually existed and
  never would
• ii.) That order could be obtained from
  randomness.

8
Everything you need to know about probability for
this class.

• Basic principle The probability of any
  particular event is equal to the ratio of the
  number of ways the event can happen over the
  number of ways the event can fail to happen the
  number of ways it can happen
• To combine probabilities of independent events
  (unrelated ands), multiply the odds of each
  event.
• To combine probabilities of mutually exclusive
  events (either/ors), add the odds of each
  event.
• Bayes Rule When one probability is conditional
  on another
• P(AB) P(A and B) / P(B)

9
Basic probability theory Example 1

• A boring standard example

How likely is it that we will throw a 6 with one
dice?

• Basic principle The probability of any
  particular event is equal to the ratio of the
  number of ways the event can happen over the
  number of ways anything can happen ( the number
  of ways the event can fail to happen the number
  of ways it can happen).

10
Basic probability theory Example 2
How likely is it that we will throw a 7 with two
dice?
11
Basic probability theory Example 3
We roll the die 4 times, and never get a seven.
What is the probability that well get on the
5th roll?

• Independent events are events that dont effect
  each others probability. Since the every roll is
  independent of every other, the odds are still
  1/6.

12
Basic probability theory Example 4
We roll the die twice and get a seven both times.
What is the probability of that?

• Are the two events dependent or independent?

13
Basic probability theory Example 4
We roll the die twice and get a seven both times.
What is the probability of that?
14
Basic probability theory Example 5
We roll the die. What are the odds are getting
either a 7 or a 2?

• Now the events are mutually exclusive if one
  happens, the other cannot. To combine
  probabilities of mutually exclusive events, add
  them together.

15
Basic probability theory Example 6.1
We roll the die twice. What are the odds that we
get at least one 7 from the two rolls?

• Can we just add the probabilities of getting a 7
  on each roll?

16
Basic probability theory Example 6.2
We roll the die twice. What are the odds that we
get at least one 7 from the two rolls?

• Can we just multiply the probabilities of
  getting a 7 on each roll?

17
Basic probability theory Example 6.3
We roll the die twice. What are the odds that we
get at least one 7 from the two rolls?

• We can turn part of the problem into a problem
  of mutual exclusivity by asking what are the
  odds of there being exactly one seven out of two
  rolls?

• One way is to roll 7 first, but not second
• - The odds of this are 1/6 5/6 (independent
  events) 0.138
• Similarly, the odds of rolling 7 second are 5/6
  1/6 (independent events) 0.138
• - Since these two outcomes are mutually
  exclusive, we can add them to get 0.138 0.138
  0.277

18
Basic probability theory Example 6.4
We roll the die twice. What are the odds that we
get at least one 7 from the two rolls?

• Now we need to know what are the odds of there
  being two sevens out of two rolls?

• We already know its 1/36 0.03
• So the odds that we get at least one 7 is the
  odds of two 7s the odds of one 7 (mutually
  exclusive events, so we sum) 0.03 0.27 0.3

19
Basic probability theory The generalization

• Does anyone know how to generalize this
  calculation, so we can easily calculate the odds
  of an event of probability p happening r times
  out of n tries, for any values of p,r, and n?
• What would we get if we generalized it across all
  values of r,n with p 0.5 (coin flips)?

20
Basic probability theory Example 6.5

• We draw a card from a full deck. What is the
  probability it is either red or a face card?
• To or events A and B, we need the addition
  rule
• P(A) P(B) P(A and B)
• If we dont subtract out the cards to which both
  conditions apply, their probability is weighted
  double

21
Basic probability theory Example 6.5

• We draw a card from a full deck. What is the
  probability it is either red or a face card?
• P(A) P(B) P(A and B)
• P(red card) ???
• P(face card ???
• P(red face card) ???

22
Basic probability theory Example 6.6

• We draw a card What is the probability it is
  lower than a queen and black?
• Here it is easier to subtract out what doesnt
  apply than to add in what does

23
Basic probability theory Conditional probability

• What if a dice is biased? How can we deal with
  relevant prior probabilities?
• P(Roll a six with cheatin dice) ? P(Roll a six
  with a fair dice)
• Why should we care about such an arcane example?
• Because real life almost always uses biased dice
• Eg. the conditional probability of being
  schizophrenic, given that a person has an
  appointment with a doctor who specializes in
  schizophrenia, is quite different from the
  unconditional probability that a person has
  schizophrenia (the base rate)

24
Conditional probability

• Conditional probability arises when one
  probability P(A) depends on another probability
  P(B) which is defined over the same population
  of events
• We say that we want to know P(A) given P(B)
  notationally, P(AB)
• We can say that the word given defines a subset
  of the population of events namely that subset
  that depends on P(B)

25
Conditional probability

• More formally, what P(AB) says is Pick out the
  events to which both P(A) and P(B) apply, and
  consider them as part of the subset of events to
  which only P(B) applies hence
• P(AB) P(A and B) / P(B)
• P(A,B) / P(B) A notational change only

26
Example Probability of cancer given that youre
female

• For example, what P(Cancer F) says is Pick out
  the people to whom both P(Cancer) and P(F) apply
  ( woman with cancer), and consider them as part
  of the subset of events to which only P(F)
  applies (e.g. woman) hence
• P(Cancer B) P(Cancer and F) / P(F)

Have skin cancer
Women
Woman with skin cancer
27
Example Probability of cancer given that youre
female

• All we are doing with P(Cancer F) P(Cancer
  and F) / P(F) is selecting out the same subset of
  the event population in both the first and second
  parts of the conditional in this case, only
  women

Have skin cancer
Women
Woman with skin cancer
28
Example Probability of cancer given that youre
female

• In other words, we are merely asking What
  proportion of woman have cancer? P(Cancer F)
  P(Cancer and F) / P(F)

Have skin cancer
Women
Woman with skin cancer
29
Conditional probability

• What we are doing with P(AB) P(A and B) / P(B)
  is selecting out the same subset of the event
  population in both the numerator and the
  denominator
• The reason it is so important to do this is
  because in some cases P(A) in some relevant
  subset of an event population is very different
  from P(A) in the whole event population
• For example, P(Have healthcare Canadian) is
  very different from P(Have healthcare)
• Everyone in Canada has healthcare P(Have
  healthcare Canadian) 1
• Most people in the world do not have healthcare
  P(Have healthcare)

30
A generalization Bayes' rule

• P(AB) P(BA) P(A) / P(B)
• Some people find Bayes' rule helpful, since in
  some cases it can clarify the problem being
  considered
• Some people find it more confusing than helpful,
• The important point to understand is that Bayes'
  rule is just a re-statement of the definition of
  conditional probability, not a new finding
• Note Multivariate version of Bayes' rule are
  defined we dont use them in this class, in
  which we only consider two-possibility problems.

31
A generalization Bayes' rule

• P(AB) P(BA) P(A) / P(B)
• Note that all this says is P(A and B) P(BA)
  P(A) because we already saw that P(AB) P(A
  and B) / P(B)
• So The probability of having being a Canadian
  and having health care is the same as the
  probability of having health care given that you
  are Canadian (1) X the probability of being
  Canadian (0.004)
• If only females had health care, then the
  probability of being Canadian and having health
  care would be half as much the probability of
  having health care given that you are Canadian
  (0.5) X the probability of being Canadian (0.004)

32
A generalization Bayes' rule

• P(AB) P(BA) P(A) / P(B)
• Note that all this says is P(A and B) P(BA)
  P(A)
• Another intuitive example The probability of
  winning the lottery and buying a ticket is
  dependent both on the probability of winning the
  lottery when given that you have a ticket and
  on the probability that you buy a ticket
• A person with a low probability of buying a
  lottery ticket has a lower probability of holding
  a winning ticket than a person that buys more
  tickets even though P(WinningTicket)- the odds
  of any particular ticket being a winning ticket-
  is the same for both people

33
A generalization Bayes' rule
P(AB) P(BA) P(A) / P(B) Proof By
definition, (1.) P(AB) P(A,B) / P(B) (2.)
P(BA) P(A,B) / P(A) (3.) P(AB) P(B) P(A,B)
Multiply (1.) by P(A) (4.) P(BA) P(A) P(A,B)
Multiply (2.) by P(A) (5.) P(AB) P(B)
P(BA) P(A) Substitute (4.) in (3.) (6.) P(AB)
P(BA) P(A) / P(B) Divide by P(B)
34
One more example

• P(Female Second row)
• The relevant numbers are something like this
• in class 35
• Females in class 30
• Number in row 2 7
• Males in row 2 2
• Female in row 2 5

35
Another example

• The relevant numbers are something like this
• in class 35
• Females in class 30
• Number in row 2 7
• Males in row 2 2
• Females in row 2 5
• P (Female and Row 2) 5/35
• P(R2) 7/35
• P(F2) P(F R2) / P(R2)
• 5/35 / 7/35
• 5/7

36
The failings of Harvard Medical School

• A particular disorder has a base rate occurrence
  of 1/1000 people. A test to detect this disease
  has a false positive rate of 5. Assume that the
  test diagnoses correctly every person who has the
  disease. What is the chance that a randomly
  selected person with a positive result actually
  has the disease? Take a guess.

37
The failings of Harvard Medical School

• A particular disorder has a base rate occurrence
  of 1/1000 people. A test to detect this disease
  has a false positive rate of 5. Assume that the
  test diagnoses correctly every person who has the
  disease. What is the chance that a randomly
  selected person with a positive result actually
  has the disease?

• Conditional probability P(AB) P(A,B) / P(B)
  P(A and B)/P(B)
• Let A Has the disorder and B Has a positive
  test result
• In 10,000 people, P(A and B) See the answer
  in class
• P(B) See the answer in class
• Chance that a randomly selected person with a
  positive result actually has the disease See
  the answer in class

38
Lets try Bayes' rule
P(AB) P(BA) P(A) / P(B) Let P(A)
Probability of disease 0.001 P(B)
Probability of positive test 0.05 0.001
0.0501 P(BA) Probability of positive
test given disease 1 Then P(AB) P(BA)
P(A) / P(B)
39
The Notorious 3-Curtain (Lets Make A Deal)
Problem!

• Three curtains hide prizes. One is good. Two are
  not.
• You choose a curtain.
• The MC opens another curtain. Its not good.
• He gives you the chance to stay with our first
  choice, or switch to the remaining unopened
  curtain.
• Should you stay or switch, or does it matter?