1'4 Joint and Conditional Probability

1
1.4 Joint and Conditional Probability

• Joint Probability
• The probability P(A?B) for two events A and B
  which intersect in the sample space
• Conditional Probability
• Given some event B with nonzero probability
• Define the conditional probability of an event A,
• If A and B are mutually exclusive,

2
1.4 Joint and Conditional Probability

• Axioms of conditional probability
• Axiom 1
• Axiom 2
• Axiom 3 if A and C are mutually exclusive

3
1.4 Joint and Conditional Probability

• (Ex1.4.1) Define three events
• A Draw a 47? resistor
• B Draw a resistor with 5 tolerance
• C Draw a 100? resistor

4
1.4 Joint and Conditional Probability
5

• Total Probability
• Expressing P(A) of any event A defined on a
  sample space S in terms of conditional
  probabilities
• Given N mutually exclusive events Bn, n1, 2,,
  N.
• Total probability of event A
• Proof using and

6
(No Transcript)
7

• Bayes Theorem
• Let Bn be the mutually exclusive events
• Using

8
Ex1.4-2 Binary Symmetric Channel

• Consider elementary binary (0 or 1) comm.
  systems. The channel occasionally causes errors
• Sample space two elements 0 or 1

9
Ex1.4-2 Binary Symmetric Channel

• P(A1Bi)P(A2Bi) 1 A1 and A2 are mutually
  exclusive
• From total probability

a prior probabilities
transition probabilities
10
Ex1.4-2 Binary Symmetric Channel

• Posterior probability (????)
• Error probability

Probabilities of correct system transmission
Probabilities of system error
11
1.5 Independent Events

• Two events
• Consider two events A, B where P(A)?0, P(B)?0
• Statistically independent
• if the probability of occurrence of one event is
  not affected by the occurrence of the other event
• For statistically independent events
• For mutually exclusive events
• In order for two events to be independent they
  must have an intersection A?B ? ?
• Papoulis pp.40 ex. 2-21.

12
1.5 Independent Events

• Comparison of mutually exclusive and independent
  events

13
1.5 Independent Events

• Multiple Events
• Consider three events A1, A2, A3
• For N events A1, A2, , AN , 1? i lt j lt k lt ? N

Statistically independent if all conditions above
are satisfied
14
1.5 Independent Events

• Papoulis pp. 40 ex. 2-21
• 3 switches connected in parallel operate
  independently
• Probability p with which each switch remains
  closed
• (a) Probability of receiving an input signal at
  the output
• Sol-1) event Ai Si is closed ? P(Ai) p,
  i1,2,3
• event R input signal is received at the
  output
• ?

15
1.5 Independent Events
De Morgans Law
independent event

• Sol-2) using total probability

Lets select the first relation
16
1.5 Independent Events

• (b) Find the prob. that S1 is open, given that an
  input signal is received at the output

17
Ex. 1.5-2

• Drawing four cards from an ordinary 52-card deck
• Let events A1, A2, A3, A4 define drawing an ace
  on the first, second, third and forth cards,
  respectively
• Draw the cards assuming each is replaced after
  the draw
• Suppose we keep each card after it is drawn

18
1.5 Independent Events

• If N events A1, A2,, AN are independent, then
  any one of them is independent of any event
  formed by unions, intersections, and complements
  of the others
• For two independent events A1 and A2,
• A1 is independent of
• is independent of A2
• is independent of
• For three independent events A1, A2, and A3
• Any one is independent of the joint occurrence of
  the other two
• Any one event is also independent of the union of
  the other two

19
1.6 Combined Experiments

• A combined experiment consists of forming a
  single experiment by suitably combining
  individual experiments
• Combined Sample Space
• Consider only two sub-experiments
• Let S1 and S2 be the sample spaces of the two
  sub-experiments
• Let s1 and s2 be the elements of S1 and S2
• Form a new space S, combined sample space
• Elements are ordered pairs (s1, s2)
• S1 M elements S2 N elements ? S MN elements
• The combined sample space

20
1.6 Combined Experiments

• Example 1.6-1
• S1 flip a coin
• S2 roll a single die
• Example 1.6-2 flip a coin twice
• For N sample spaces Sn, n1,2,N, having elements
  sn

21
1.6 Combined Experiments

• Events on the Combined Space
• Consider two sub-experiments with sample space S1
  and S2
• A any event on S1
• B any event on S2
• An event on S consisting of all pairs (s1,s2)
• Elements of A elements of the event A?S2
  defined on S, and elements of B elements of the
  event B?S1 defined on S

22
1.6 Combined Experiments

• Example 1.6-3
• Combined sample space
• For events
• where

23
1.6 Combined Experiments

• Probabilities
• For N independent experiments An and An?Sn, n
  1, 2,, N
• Permutations(??)

24
1.6 Combined Experiments

• Combinations(??)
• The number of r things taken from n things
• (ex) draw out 3 of A,B,C,D,and E
• Permutations ABC, ACB, BAC, BCA, CAB, and CBA
  are different events
• Combinations 6 events above are same events
  ?combinationspermutations/r!, where r is the
  number of trials

25
1.7 Bernoulli Trials

• Consider any experiment for which there are only
  two possible outcomes (A or ) on any trial
• Bernoulli trials
• Repeat the basic experiment N times
• Determine the probability that A is observed
  exactly k times out of the N trials
• Assume that elementary events are statistically
  independent for every trial
• Let
• After N trials. A k times, N-k times

k terms
N-k terms
26
Example 1.7-1

• A submarine attempts to sink an aircraft carrier
• Success if 2 or more torpedoes hit the carrier
• The submarine fires 3 torpedoes
• Hit probability 0.4
• (prob) What is the prob. that the carrier will be
  sunk?
• Define the event A torpedo hits
• P(A) 0.4, N 3
• Using the result of Bernoulli trials

27
Example 1.7-1

• Ex We place at random n points in the interval
  0, T. What is the prob. that k of n points are
  in t1, t2?

28
1.7 Bernoulli Trials

• Stirlings formula
• When N, k, (N-k) are large, the factorials are
  difficult to evaluate ? approximations become
  useful
• De Moivre-Laplace approximation
• if N, k, and (N-k) are large, and k is near
  Np such that its deviations from Np (higher or
  lower) are small in magnitude relative to both Np
  and N(1-p)

Gaussian
29
1.7 Bernoulli Trials

• Possion approximation for large N and small p
• (Ex. 1.7-3)