CSE 321 Discrete Structures

1
CSE 321 Discrete Structures

• Winter 2008
• Lecture 19
• Probability Theory

TexPoint fonts used in EMF. Read the TexPoint
manual before you delete this box. AAAAA
2
Announcements

• Readings
• Probability Theory
• 6.1, 6.2 (5.1, 5.2) Probability Theory
• 6.3 (New material!) Bayes Theorem
• 6.4 (5.3) Expectation
• Advanced Counting Techniques Ch 7.
• Not covered

3
Discrete Probability

• Experiment Procedure that yields an outcome
• Sample space Set of all possible outcomes
• Event subset of the sample space

S a sample space of equally likely outcomes, E
an event, the probability of E, p(E) E/S
4
Example Dice
Sample space, event, example
5
Example Poker
Probability of 4 of a kind
6
Combinations of Events
EC is the complement of E P(EC) 1 P(E)
P(E1? E2) P(E1) P(E2) P(E1? E2)
7
Combinations of Events
EC is the complement of E P(EC) 1 P(E)
P(E1? E2) P(E1) P(E2) P(E1? E2)
8
Probability Concepts

• Probability Distribution
• Conditional Probability
• Independence
• Bernoulli Trials / Binomial Distribution
• Random Variable

9
Discrete Probability Theory

• Set S
• Probability distribution p S ? 0,1
• For s ? S, 0 ? p(s) ? 1
• ?s? S p(s) 1
• Event E, E? S
• p(E) ?s? Ep(s)

10
Examples
11
Conditional Probability
Let E and F be events with p(F) gt 0. The
conditional probability of E given F, defined by
p(E F), is defined as
12
Examples
Flip a coin 5 times, W is the event of three or
more heads
13
Independence
The events E and F are independent if and only if
p(E? F) p(E)p(F)
E and F are independent if and only if p(E F)
p(E)
14
Are these independent?

• Flip a coin three times
• E the first coin is a head
• F the second coin is a head
• Roll two dice
• E the sum of the two dice is 5
• F the first die is a 1
• Roll two dice
• E the sum of the two dice is 7
• F the first die is a 1
• Deal two five card poker hands
• E hand one has four of a kind
• F hand two has four of a kind

0.0000000576 0.0000000740
15
Bernoulli Trials and Binomial Distribution

• Bernoulli Trial
• Success probability p, failure probability q

The probability of exactly k successes in n
independent Bernoulli trials is
16
Random Variables
A random variable is a function from a sample
space to the real numbers
17
Bayes Theorem
Suppose that E and F are events from a sample
space S such that p(E) gt 0 and p(F) gt 0. Then
18
False Positives, False Negatives
Let D be the event that a person has the
disease Let Y be the event that a person tests
positive for the disease
19
Testing for disease
Disease is very rare p(D) 1/100,000 Testing
is accurate False negative 1 False positive
0.5 Suppose you get a positive result, what do
you conclude?
P(YCD) P(YDC)
20
P(DY)
Answer is about 0.002
21
Spam Filtering
From Zambia Nation Farmers Union
znfukabwe_at_mail.zamtel.zm Subject Letter of
assistance for school installation To Richard
Anderson Dear Richard, I hope you are fine, Iam
through talking to local headmen about the
possible assistance of school installation. the
idea is and will be welcome. I trust that you
will do your best as i await for more from
you. Once again Thanking you very much Sebastian
Mazuba.
22
Bayesian Spam filters

• Classification domain
• Cost of false negative
• Cost of false positive
• Criteria for spam
• v1agra, ONE HUNDRED MILLION USD
• Basic question given an email message, based on
  spam criteria, what is the probability it is spam

23
Email message with phrase Account Review

• 250 of 20000 messages known to be spam
• 5 of 10000 messages known not to be spam
• Assuming 50 of messages are spam, what is the
  probability that a message with Account Review
  is spam

24
Proving Bayes Theorem
25
Expectation
The expected value of random variable X(s) on
sample space S is
26
Flip a coin until the first headExpected number
of flips?
Probability Space Computing the expectation
27
Linearity of Expectation
E(X1 X2) E(X1) E(X2) E(aX) aE(X)
28
Hashing
H M ? 0..n-1
If k elements have been hashed to random
locations, what is the expected number of
elements in bucket j? What is the expected
number of collisions when hashing k elements to
random locations?
29
Hashing analysis
Sample space 0..n-1 ? 0..n-1 ? . . . ?
0..n-1
Random Variables Xj number of elements hashed
to bucket j C total number of collisions Bij
1 if element i hashed to bucket j Bij 0 if
element i is not hashed to bucket j Cab 1 if
element a is hashed to the same bucket as element
b Cab 0 if element a is hashed to a different
bucket than element b
30
Counting inversions
Let p1, p2, . . . , pn be a permutation of 1 . .
. n pi, pj is an inversion if i lt j and pi gt pj
4, 2, 5, 1, 3 1, 6, 4, 3, 2, 5 7, 6, 5, 4, 3,
2, 1
31
Expected number of inversions for a random
permutation
32
Insertion sort
4
2
5
1
3
for i 1 to n-1 j i while (j gt
0 and A j - 1 gt A j ) swap(A j -1, A j
) j j 1
33
Expected number of swaps for Insertion Sort
34
Left to right maxima
max_so_far A0 for i 1 to n-1 if
(A i gt max_so_far) max_so_far A i
5, 2, 9, 14, 11, 18, 7, 16, 1, 20, 3, 19, 10, 15,
4, 6, 17, 12, 8
35
What is the expected number of left-to-right
maxima in a random permutation