Chap 3 Conditional Probability and Independence Ghahramani 3rd edition

1
Chap 3 Conditional Probability and
IndependenceGhahramani 3rd edition
2
Outline

• 3.1 Conditional probability
• 3.2 Law of multiplication
• 3.3 Law of total probability
• 3.4 Bayes formula
• 3.5 Independence
• 3.6 Applications of probability to genetics

3
3.1 Conditional probability

• Def If P(B)gt0, the conditional probability of A
  given B, denoted by P(AB), is

4
Conditional probability

• Ex 3.1 In a certain region of Russia,
• P(one lives at least 80)0.75,
• P(one lives at least 90)0.63.
• What is the probability that a
  randomly selected 80-year-old person from this
  region will survive to become 90?
• Sol Let B and A be the events that the person
  selected survives to become 80 and 90 years old,
  respectively.
• P(AB)P(AB)/P(B)P(A)/P(B)0.63/0.750.84

5
Conditional probability

• Ex 3.2 From the set of all families with 2
  children, a family is selected at random and is
  found to have a girl. What is the probability
  that the other child of the family is a girl?
  Assume that in a 2-child family all sex
  distributions are equally probable.
• Sol Let B and A be the events that the family
  has a girl (meaning at least a girl) and the
  family has 2 girls, respectively.
• possible equally likely outcomes
  (b,b),(b,g),(g,b),(g,g)
• P(AB)P(AB)/P(B)P(A)/P(B)(1/4)/(3/4)

6
Conditional probability

• Ex 3.5 Draw 8 cards from a deck of 52 cards.
  Given 3 of them are spades, P(the remaining 5 are
  also spades)?
• Sol B the event that at least 3 of them are
  spades
• A the event that all 8 cards selected
  are spades
• P(AB)P(AB)/P(B)P(A)/P(B)
• P(A)C(13,8)/C(52,8)
• P(B)C(13,3)C(39,5)/C(52,8)
• C(13,4)C(39,4)/C(52,8)
• C(13,8)C(39,0)/C(52,8)
• P(AB)5.44 x 10-6

7
3.2 Law of multiplication

8
Law of multiplication

• Ex 3.9 Suppose 5 good fuses and 2 defective ones
  have been mixed up. To find the defective fuses,
  we test them 1-by-1, at random and without
  replacement. What is the probability that we are
  lucky and find both of the defective fuses in the
  first two tests?
• Sol Let D1 and D2 be the events of finding a
  defective fuse in the 1st and 2nd tests,
  respectively.
• P(D1D2)P(D1)P(D2D1)
• 2/7x1/61/21

9
Law of multiplication

• Thm 3.2 If P(A1A2A3 An-1)gt0, then
• P(A1A2A3 An-1 An)
• P(A1)P(A2A1)P(A3A1A2) P(AnA1A2A3
  An-1)
• Proof For n3, if P(AB)gt0,
• then P(ABC)P(AB)P(CAB) and P(A)gt0
• then P(AB)P(A)P(BA)
• so P(ABC) P(A)P(BA)P(CAB)
• Thm can be proved by math induction.

10
Law of multiplication

• Ex 3.11 Suppose 5 good fuses and 2 defective ones
  have been mixed up. To find the defective fuses,
  we test them 1-by-1, at random and without
  replacement. What is the probability that we are
  lucky and find both of the defective fuses in
  exactly three tests?

11
Law of multiplication

• Sol Let D1 , D2 and D3 be the events that the
  1st, 2nd, and 3rd fuses tested are defective,
  respectively.
• Let G1 , G2 and G3 be the events that the
  1st, 2nd, and 3rd fuses tested are good,
  respectively.
• P(G1D2D3UD1G2D3)
• P(G1D2D3)P(D1G2D3)
• P(G1)P(D2G1)P(D3G1D2)
• P(D1)P(G2D1)P(D3D1G2)
• 5/7x2/6x1/52/7x5/6x15
• 0.095

12
3.3 Law of total probability

• Thm 3.3(Law of total probability)
• Let P(B)gt0, and P(Bc)gt0, then
• P(A)P(AB)P(B)P(ABc)P(Bc)
• Proof By Thm 1.7,
• P(A)P(AB)P(ABc)
• and use law of multiplication.

13
Law of total probability

• Ex 3.12 An insurance company rents 35 of the
  cars for its customers from agency I and 65 from
  agency II. If 8 of the cars of agency I and 5
  of the cars of agency II break down during the
  rental periods, what is the probability that a
  car rented by this insurance company breaks down?
• Sol P(A)P(AI)P(I) P(AII)P(II)
• (0.08)(0.35)(0.05)(0.65)0.0605

14
Law of total probability

• Ex 3.14(Gamblers ruin problem)
• 2 gamblers play the game of heads or
  tails, in which each time a fair coin lands
  heads up, player A wins 1 from B, and each time
  it lands tails up, player B wins 1 from A.
  Suppose that the player A initially has a dollars
  and player B has b dollars. If they continue to
  play this game successively, what is the
  probability that
• (a) A will be ruined
• (b) the game goes forever with nobody
  winning?

15
Law of total probability

• Sol (a)
• Let E be the event that A will be ruined if
  he starts with i dollars, and let piP(E). Our
  aim is to calculate pa. To do so we define F to
  be the event that A wins the 1st game. Then
• P(E)P(EF)P(F)P(EFc)P(Fc)
• gt pi pi1(1/2)pi-1(1/2)
• gt pi1 pi pi pi-1

16
Law of total probability

• p01, pab0, and let p1- p0 x
• pi1 pi pi pi-1 p2- p1 p1- p0
  x
• Thus p1 p0 x
• p2 p0 2x
• pi p0 ix
• and pab p0 (ab)x gt x -1/(ab)
• So pi 1 i/(ab). In particular pa
  b/(ab)

17
Law of total probability

• (b)
• The same method can be used with obvious
  modifications to calculate qi, the probability
  that B is ruined if he starts with i dollars.
• qi 1 i/(ab)
• Since B starts with b dollars, he will be
  ruined with probability qb a/(ab).
• Thus the probability that the game goes on
  forever with nobody winning is 1-(qbpa)0.

18
Law of total probability

• Definition Let B1, B2, , Bn be a set of
  nonempty subsets of the sample space S of an
  experiment. If the events B1, B2, , Bn are
  mutually exclusive and ,
  the set
• B1, B2, , Bn is called a partition of S.

19
Law of total probability

• Thm 3.4(Law of total probability)
• If B1, B2, , Bn is a partition of the
  sample space S of an experiment and P(Bi)gt0 for
  all i. Then for any event A of S,
• P(A)P(AB1)P(B1)P(AB2)P(B2)
• P(ABn)P(Bn)
• Thm 3.5 Thm 3.4 can be generalized to n

20
Law of total probability

• Ex 3.15 Suppose that 80 of the seniors, 70 of
  the juniors, 50 of the sophomores, and 30 of
  the freshmen of a college use the library of
  their campus frequently. If 30 of all students
  are freshmen, 25 are sophomores, 25 are
  juniors, and 20 are seniors, what percent of all
  students use the library frequently?
• Sol P(A)P(AF)P(F)P(AO)P(O)
• P(AJ)P(J)P(AE)P(E)
• (0.30)(0.30)(0.50)(0.25)
• (0.70)(0.25)(0.80)(0.20)
• 0.55

21
3.4 Bayes formula

• In a bolt factory, 30, 50, and 20 of production
  is manufactured by machines I, II, and III,
  respectively. If 4, 5, and 3 of the output of
  these respective machines is defective, what is
  the probability that a randomly selected bolt
  that is found to be defective is manufactured by
  machine III?

22
Bayes formula

• To solve this problem, let A be the event
  that a random bolt is defective and Bi be the
  event that it is manufactured by machine i.
• We are asked to find P(B3A).
• Now P(B3A) P(B3A)/P(A)
• and P(B3A)P(AB3)P(B3)
• What is P(A)? By law of total probability
• P(A)P(AB1)P(B1)P(AB2)P(B2)P(AB3)P(B3)

23
Bayes formula

• So Bayes formula
• P(B3A)
• P(B3A)/P(A)
• P(AB3)P(B3)/ P(AB1)P(B1)P(AB2)P(B
  2)P(AB3)P(B3)

24
Bayes formula

• Ex 3.19 A box contains 7 red and 13 blue balls.
  2 balls are selected at random and are discarded
  without their colors being seen. If a 3rd ball
  is drawn randomly and observed to be red, what is
  the probability that both of the discarded balls
  were blue?

25
Bayes formula

• Sol Let BB, BR, and RR be the events that the
  discarded balls are blue and blue, blue and red,
  red and red, respectively. By Bayesformula
• P(BBR)P(RBB)P(BB)/
• P(RBB)P(BB)P(RBR)P(BR)
• P(RRR)P(RR)

26
Bayes formula

• Now
• P(BB)(13/20)(12/19)39/95
• P(RR)(7/20)(6/19)21/190
• P(BR)(13/20)(7/19)(7/20)(13/19)91/190
• Thus P(BBR)(7/18)(39/95)/
• (7/18)(39/95)(6/18)(91/190)(5/18)(21/190)0.46
  

27
3.5 Independence

• In general, the conditional probability of A
  given B is not the probability of A. However, if
  it is, that is P(AB)P(A), we say that A is
  independent of B.
• P(AB)P(A)
• P(AB)/P(B)P(A)
• P(AB)P(A)P(B)
• Definition Two events A and B are called
  independent if
• P(AB)P(A)P(B)

28
Independence

• Ex 3.24 An urn contains 5 red and 7 blue balls.
  Suppose that 2 balls are selected at random and
  with(without) replacement. Let A and B be the
  events that the first and the second ball are
  red, respectively. Are A and B independent?

29
Independence

• Sol (a) with replacement
• P(AB)P(A)P(B)(5/12)(5/12)
• So A and B are independent
• (b) without replacement
• P(BA)4/11 while
• P(B)P(BA)P(A)P(BAc)P(Ac)
• (4/11)(5/12)(5/11)(7/12)5/
  12
• So A and B are dependent

30
Independence

• Thm 3.6 If A and B are independent, then A and Bc
  are independent as well.
• Proof P(A)P(AB)P(ABc) Therefore
• P(ABc)P(A)-P(AB)
• P(A)-P(A)P(B)
• P(A)1-P(B)
• P(A)P(Bc)
• Coro If A and B are independent, then Ac and Bc
  are independent as well.

31
Independence

• Ex 3.28(Jailers paradox)
• The jailer of a prison in which Alex, Bill, and
  Tim are held is the only person, other than the
  judge, who knows which of these prisoners is
  condemned to death, and which two will be freed.
  Alex has written a letter to his fiancee and
  wants to give it to either Bill or Tim, whoever
  goes free, to deliver. So Alex asks the jailer
  to tell him which of the two will be freed. The
  jailer refuses to give that information to Alex,
  explaining that, if he does, the probability of
  Alex dying increases from 1/3 to 1/2. Really?

32
Independence

• No. The jailer is wrong!
• We will show that even if the jailer
  tells the name of which of the other two who will
  go free, the probability of Alex dying is still
  1/3.
• Sol-1
• Let A, B, and T be the events that Alex
  dies,
• Bill dies, and Tim dies. Let
• w1(T, the jailer tells Alex that Bill
  goes free)
• w2(B, the jailer tells Alex that Tim goes
  free)
• w3(A, the jailer tells Alex that Bill
  goes free)
• w4(A, the jailer tells Alex that Tim goes
  free)

33
Independence

• The sample space S w1, w2, w3, w4
• P(w1)P(w2)1/3
• P(w3)P(w4)1/6
• Let J be the event that the jailer tells Alex
  that Tim goes free then

34
Independence

• Sol-2(Zweifel, in June 1986 issue of Mathematics
• Magazine, page 156)
• He analyzes this paradox by using Bayes formula

35
Independence

• We now extend the concept of independent to
  finitely many events.
• Definition The events A, B, and C are called
  independent if
• P(AB)P(A)P(B)
• P(AC)P(A)P(C)
• P(BC)P(B)P(C)
• P(ABC)P(A)P(B)P(C)

36
Independence

• Definition The set of events A1, A2, , An is
  called independent if for every subset Ai1, Ai2,
  , Aik, kgt1,
• P(Ai1 Ai2 Aik)P(Ai1)P(Ai2)P(Aik)

37
Independence

• Ex 3.31 We draw cards, one at a time, at random
  and successively from an ordinary deck of 52
  cards with replacement. What is the probability
  that an ace appears before a face card?

38
Independence

• Sol
• Let E be the event of an ace appearing before a
  face card. Let A, F, and B be the events of ace,
  face card, and neither in the first experiment,
  respectively. Then
• P(E)P(EA)P(A)P(EF)P(F)P(EB)P(B)
• 1(4/52)0(12/52)P(EB)(36/52)
• P(EB)P(E) so P(E)4/52P(E)(36/52)
• so P(E)1/4

39

• Skip
• 3.6 Applications of probability to genetics