Mathematics topic handout: Conditional probability

1
Conditional Probability and Bayes
Theorem Consider two events A and B. Each can
either occur or not occur. Event A not occurring
is denoted by A and B not occurring is denoted
by B. We can construct TWO tree diagrams to map
out the possible permutations of outcomes. The
probability of A occurring given B has occurred
is P(AB). Events A and B occurring is P(A,B).
Note the order does not matter for the latter.
B
A
B
A
B
A
B
A
B
A
B
A
Now P(A,B) P(B,A), hence
This is called Bayes Theorem, and allows P(BA)
to be computed from P(AB), P(A) and P(B)
Mathematics topic handout Conditional
probability Bayes Theorem Dr Andrew French.
www.eclecticon.info PAGE 1
2
Conditional probability example P(AB) 1/3,
P(A) 1/4 and P(B) 1/5 Find all the
other probabilities in both tree diagrams
corresponding to events A and B .
A
B
B
A
A
B
A
B
B
A
A
B
Mathematics topic handout Conditional
probability Bayes Theorem Dr Andrew French.
www.eclecticon.info PAGE 2
3
Consider a medical test (Pass, Y or Fail, N) for
a disease. The probability of passing (or indeed
failing) is conditional upon whether the
patient actually has the disease (true, T) or not
(false, F). Unfortunately the latter is what we
want to infer from the test, not the other way
round. Tests are never perfect, and there will
be four possible outcomes The middle
two options are obviously undesirable, and often
ignored by medical practitioners.
Thomas Bayes 1701-1761
Possible outcome Conditional probability
Pass test given person actually has the disease P(YT) t
Fail test given person actually has the disease (false negative) P(NT) 1-t
Pass test given person doesnt actually have the disease (false positive) P(YF) q
Fail test given person doesnt actually have the disease P(NF) 1-q
Probabilities we are really interested in
Reality (or model thereof). We could work this
out from historical statistics
Obviously these two views are equivalent, hence
This is Bayes Theorem
Now note
Bayes Theorem
Mathematics topic handout Conditional
probability Bayes Theorem Dr Andrew French.
www.eclecticon.info PAGE 3
4
Example Testing a member of the public at random
for a disease. The chance of having disease is p
1/1000 The disease test is 95 accurate. i.e.
t 1 - q 0.95, and symmetric! (i.e. false
positive is equally unlikely as a false negative)
Probability of having the disease given a
positive test result is
Note if q 1 - t
i.e. because of the low probability of actually
having the disease in the first place the overall
probability of someone having the disease given a
positive test result is actually very low! To
give a better than 95 accurate result, our test
accuracy t must be t 0.999/(0.001/0.95 1)
99.8 accurate.
Example from The Signal and the Noise by Nate
Silver p247
PRIOR PROBABILITY PRIOR PROBABILITY PRIOR PROBABILITY
Initial estimate of how likely it is that terrorists would crash planes into Manhattan skyscrapers p 0.005
A NEW EVENT OCCURS FIRST PLANE HITS WORLD TRADE CENTER A NEW EVENT OCCURS FIRST PLANE HITS WORLD TRADE CENTER A NEW EVENT OCCURS FIRST PLANE HITS WORLD TRADE CENTER
Probability of plane hitting if terrorists are attacking Manhattan skyscrapers t 100
Probability of plane hitting if terrorists are not attacking Manhattan skyscrapers (i.e. an accident) q 0.008
POSTERIOR PROBABILITY POSTERIOR PROBABILITY POSTERIOR PROBABILITY
Revised estimate of probability of terror attack, given first plane hitting World Trade Center 38
But then probability of terror attack, given
second plane hitting the World Trade Center is
99.99 since we re-do the analysis but set p 38
Mathematics topic handout Conditional
probability Bayes Theorem Dr Andrew French.
www.eclecticon.info PAGE 4
5
Cluedo example
PRIOR PROBABILITY PRIOR PROBABILITY PRIOR PROBABILITY
Probability of Colonel Mustard being of murderous intent p 5
CONDITIONAL PROBABILITIES LIKELIHOODS CONDITIONAL PROBABILITIES LIKELIHOODS CONDITIONAL PROBABILITIES LIKELIHOODS
Probability of Professor Plum meeting his doom given Colonel Mustard is a potential murderer t 50
Probability of Professor Plum dying given Colonel Mustard is not feeling particularly murderous. i.e. he dies via natural causes, or someone else kills him.... q 1
POSTERIOR PROBABILITY POSTERIOR PROBABILITY POSTERIOR PROBABILITY
Probability of Colonel Mustard being the murderer of Professor Plum, given Professor Plum is observed to be dead P ?
Mathematics topic handout Conditional
probability Bayes Theorem Dr Andrew French.
www.eclecticon.info PAGE 5
6
Mathematics topic handout Conditional
probability Bayes Theorem Dr Andrew French.
www.eclecticon.info PAGE 6