Chap 3 Conditional Probability and

IndependenceGhahramani 3rd edition

Outline

- 3.1 Conditional probability
- 3.2 Law of multiplication
- 3.3 Law of total probability
- 3.4 Bayes formula
- 3.5 Independence
- 3.6 Applications of probability to genetics

3.1 Conditional probability

- Def If P(B)gt0, the conditional probability of A

given B, denoted by P(AB), is

Conditional probability

- Ex 3.1 In a certain region of Russia,
- P(one lives at least 80)0.75,
- P(one lives at least 90)0.63.
- What is the probability that a

randomly selected 80-year-old person from this

region will survive to become 90? - Sol Let B and A be the events that the person

selected survives to become 80 and 90 years old,

respectively. - P(AB)P(AB)/P(B)P(A)/P(B)0.63/0.750.84

Conditional probability

- Ex 3.2 From the set of all families with 2

children, a family is selected at random and is

found to have a girl. What is the probability

that the other child of the family is a girl?

Assume that in a 2-child family all sex

distributions are equally probable. - Sol Let B and A be the events that the family

has a girl (meaning at least a girl) and the

family has 2 girls, respectively. - possible equally likely outcomes

(b,b),(b,g),(g,b),(g,g) - P(AB)P(AB)/P(B)P(A)/P(B)(1/4)/(3/4)

Conditional probability

- Ex 3.5 Draw 8 cards from a deck of 52 cards.

Given 3 of them are spades, P(the remaining 5 are

also spades)? - Sol B the event that at least 3 of them are

spades - A the event that all 8 cards selected

are spades - P(AB)P(AB)/P(B)P(A)/P(B)
- P(A)C(13,8)/C(52,8)
- P(B)C(13,3)C(39,5)/C(52,8)
- C(13,4)C(39,4)/C(52,8)
- C(13,8)C(39,0)/C(52,8)
- P(AB)5.44 x 10-6

3.2 Law of multiplication

Law of multiplication

- Ex 3.9 Suppose 5 good fuses and 2 defective ones

have been mixed up. To find the defective fuses,

we test them 1-by-1, at random and without

replacement. What is the probability that we are

lucky and find both of the defective fuses in the

first two tests? - Sol Let D1 and D2 be the events of finding a

defective fuse in the 1st and 2nd tests,

respectively. - P(D1D2)P(D1)P(D2D1)
- 2/7x1/61/21

Law of multiplication

- Thm 3.2 If P(A1A2A3 An-1)gt0, then
- P(A1A2A3 An-1 An)
- P(A1)P(A2A1)P(A3A1A2) P(AnA1A2A3

An-1) - Proof For n3, if P(AB)gt0,
- then P(ABC)P(AB)P(CAB) and P(A)gt0
- then P(AB)P(A)P(BA)
- so P(ABC) P(A)P(BA)P(CAB)
- Thm can be proved by math induction.

Law of multiplication

- Ex 3.11 Suppose 5 good fuses and 2 defective ones

have been mixed up. To find the defective fuses,

we test them 1-by-1, at random and without

replacement. What is the probability that we are

lucky and find both of the defective fuses in

exactly three tests?

Law of multiplication

- Sol Let D1 , D2 and D3 be the events that the

1st, 2nd, and 3rd fuses tested are defective,

respectively. - Let G1 , G2 and G3 be the events that the

1st, 2nd, and 3rd fuses tested are good,

respectively. - P(G1D2D3UD1G2D3)
- P(G1D2D3)P(D1G2D3)
- P(G1)P(D2G1)P(D3G1D2)
- P(D1)P(G2D1)P(D3D1G2)
- 5/7x2/6x1/52/7x5/6x15
- 0.095

3.3 Law of total probability

- Thm 3.3(Law of total probability)
- Let P(B)gt0, and P(Bc)gt0, then
- P(A)P(AB)P(B)P(ABc)P(Bc)
- Proof By Thm 1.7,
- P(A)P(AB)P(ABc)
- and use law of multiplication.

Law of total probability

- Ex 3.12 An insurance company rents 35 of the

cars for its customers from agency I and 65 from

agency II. If 8 of the cars of agency I and 5

of the cars of agency II break down during the

rental periods, what is the probability that a

car rented by this insurance company breaks down? - Sol P(A)P(AI)P(I) P(AII)P(II)
- (0.08)(0.35)(0.05)(0.65)0.0605

Law of total probability

- Ex 3.14(Gamblers ruin problem)
- 2 gamblers play the game of heads or

tails, in which each time a fair coin lands

heads up, player A wins 1 from B, and each time

it lands tails up, player B wins 1 from A.

Suppose that the player A initially has a dollars

and player B has b dollars. If they continue to

play this game successively, what is the

probability that - (a) A will be ruined
- (b) the game goes forever with nobody

winning?

Law of total probability

- Sol (a)
- Let E be the event that A will be ruined if

he starts with i dollars, and let piP(E). Our

aim is to calculate pa. To do so we define F to

be the event that A wins the 1st game. Then - P(E)P(EF)P(F)P(EFc)P(Fc)
- gt pi pi1(1/2)pi-1(1/2)
- gt pi1 pi pi pi-1

Law of total probability

- p01, pab0, and let p1- p0 x
- pi1 pi pi pi-1 p2- p1 p1- p0

x - Thus p1 p0 x
- p2 p0 2x
- pi p0 ix
- and pab p0 (ab)x gt x -1/(ab)
- So pi 1 i/(ab). In particular pa

b/(ab)

Law of total probability

- (b)
- The same method can be used with obvious

modifications to calculate qi, the probability

that B is ruined if he starts with i dollars. - qi 1 i/(ab)
- Since B starts with b dollars, he will be

ruined with probability qb a/(ab). - Thus the probability that the game goes on

forever with nobody winning is 1-(qbpa)0.

Law of total probability

- Definition Let B1, B2, , Bn be a set of

nonempty subsets of the sample space S of an

experiment. If the events B1, B2, , Bn are

mutually exclusive and ,

the set - B1, B2, , Bn is called a partition of S.

Law of total probability

- Thm 3.4(Law of total probability)
- If B1, B2, , Bn is a partition of the

sample space S of an experiment and P(Bi)gt0 for

all i. Then for any event A of S, - P(A)P(AB1)P(B1)P(AB2)P(B2)
- P(ABn)P(Bn)
- Thm 3.5 Thm 3.4 can be generalized to n

Law of total probability

- Ex 3.15 Suppose that 80 of the seniors, 70 of

the juniors, 50 of the sophomores, and 30 of

the freshmen of a college use the library of

their campus frequently. If 30 of all students

are freshmen, 25 are sophomores, 25 are

juniors, and 20 are seniors, what percent of all

students use the library frequently? - Sol P(A)P(AF)P(F)P(AO)P(O)
- P(AJ)P(J)P(AE)P(E)
- (0.30)(0.30)(0.50)(0.25)
- (0.70)(0.25)(0.80)(0.20)
- 0.55

3.4 Bayes formula

- In a bolt factory, 30, 50, and 20 of production

is manufactured by machines I, II, and III,

respectively. If 4, 5, and 3 of the output of

these respective machines is defective, what is

the probability that a randomly selected bolt

that is found to be defective is manufactured by

machine III?

Bayes formula

- To solve this problem, let A be the event

that a random bolt is defective and Bi be the

event that it is manufactured by machine i. - We are asked to find P(B3A).
- Now P(B3A) P(B3A)/P(A)
- and P(B3A)P(AB3)P(B3)
- What is P(A)? By law of total probability
- P(A)P(AB1)P(B1)P(AB2)P(B2)P(AB3)P(B3)

Bayes formula

- So Bayes formula
- P(B3A)
- P(B3A)/P(A)
- P(AB3)P(B3)/ P(AB1)P(B1)P(AB2)P(B

2)P(AB3)P(B3)

Bayes formula

- Ex 3.19 A box contains 7 red and 13 blue balls.

2 balls are selected at random and are discarded

without their colors being seen. If a 3rd ball

is drawn randomly and observed to be red, what is

the probability that both of the discarded balls

were blue?

Bayes formula

- Sol Let BB, BR, and RR be the events that the

discarded balls are blue and blue, blue and red,

red and red, respectively. By Bayesformula - P(BBR)P(RBB)P(BB)/
- P(RBB)P(BB)P(RBR)P(BR)
- P(RRR)P(RR)

Bayes formula

- Now
- P(BB)(13/20)(12/19)39/95
- P(RR)(7/20)(6/19)21/190
- P(BR)(13/20)(7/19)(7/20)(13/19)91/190
- Thus P(BBR)(7/18)(39/95)/
- (7/18)(39/95)(6/18)(91/190)(5/18)(21/190)0.46

3.5 Independence

- In general, the conditional probability of A

given B is not the probability of A. However, if

it is, that is P(AB)P(A), we say that A is

independent of B. - P(AB)P(A)
- P(AB)/P(B)P(A)
- P(AB)P(A)P(B)
- Definition Two events A and B are called

independent if - P(AB)P(A)P(B)

Independence

- Ex 3.24 An urn contains 5 red and 7 blue balls.

Suppose that 2 balls are selected at random and

with(without) replacement. Let A and B be the

events that the first and the second ball are

red, respectively. Are A and B independent?

Independence

- Sol (a) with replacement
- P(AB)P(A)P(B)(5/12)(5/12)
- So A and B are independent
- (b) without replacement
- P(BA)4/11 while
- P(B)P(BA)P(A)P(BAc)P(Ac)
- (4/11)(5/12)(5/11)(7/12)5/

12 - So A and B are dependent

Independence

- Thm 3.6 If A and B are independent, then A and Bc

are independent as well. - Proof P(A)P(AB)P(ABc) Therefore
- P(ABc)P(A)-P(AB)
- P(A)-P(A)P(B)
- P(A)1-P(B)
- P(A)P(Bc)
- Coro If A and B are independent, then Ac and Bc

are independent as well.

Independence

- Ex 3.28(Jailers paradox)
- The jailer of a prison in which Alex, Bill, and

Tim are held is the only person, other than the

judge, who knows which of these prisoners is

condemned to death, and which two will be freed.

Alex has written a letter to his fiancee and

wants to give it to either Bill or Tim, whoever

goes free, to deliver. So Alex asks the jailer

to tell him which of the two will be freed. The

jailer refuses to give that information to Alex,

explaining that, if he does, the probability of

Alex dying increases from 1/3 to 1/2. Really?

Independence

- No. The jailer is wrong!
- We will show that even if the jailer

tells the name of which of the other two who will

go free, the probability of Alex dying is still

1/3. - Sol-1
- Let A, B, and T be the events that Alex

dies, - Bill dies, and Tim dies. Let
- w1(T, the jailer tells Alex that Bill

goes free) - w2(B, the jailer tells Alex that Tim goes

free) - w3(A, the jailer tells Alex that Bill

goes free) - w4(A, the jailer tells Alex that Tim goes

free)

Independence

- The sample space S w1, w2, w3, w4
- P(w1)P(w2)1/3
- P(w3)P(w4)1/6
- Let J be the event that the jailer tells Alex

that Tim goes free then

Independence

- Sol-2(Zweifel, in June 1986 issue of Mathematics
- Magazine, page 156)
- He analyzes this paradox by using Bayes formula

Independence

- We now extend the concept of independent to

finitely many events. - Definition The events A, B, and C are called

independent if - P(AB)P(A)P(B)
- P(AC)P(A)P(C)
- P(BC)P(B)P(C)
- P(ABC)P(A)P(B)P(C)

Independence

- Definition The set of events A1, A2, , An is

called independent if for every subset Ai1, Ai2,

, Aik, kgt1, - P(Ai1 Ai2 Aik)P(Ai1)P(Ai2)P(Aik)

Independence

- Ex 3.31 We draw cards, one at a time, at random

and successively from an ordinary deck of 52

cards with replacement. What is the probability

that an ace appears before a face card?

Independence

- Sol
- Let E be the event of an ace appearing before a

face card. Let A, F, and B be the events of ace,

face card, and neither in the first experiment,

respectively. Then - P(E)P(EA)P(A)P(EF)P(F)P(EB)P(B)
- 1(4/52)0(12/52)P(EB)(36/52)
- P(EB)P(E) so P(E)4/52P(E)(36/52)
- so P(E)1/4

- Skip
- 3.6 Applications of probability to genetics