DISCRETE MATHEMATICS Lecture 18

1
DISCRETE MATHEMATICSLecture 18

• Dr. Kemal Akkaya
• Department of Computer Science

2
6.1-2 Why Probability?

• In the real world, we often dont know whether a
  given proposition is true or false.
• Probability theory gives us a way to reason about
  propositions whose truth is uncertain.
• It is useful in weighing evidence, diagnosing
  problems, and analyzing situations whose exact
  details are unknown.
• Many CS applications Networking, randomized
  algorithms

3
Experiments Sample Spaces

• When one performs an experiment such as tossing a
  single fair coin, rolling a single fair dice, or
  selecting two students at random from a class of
  20 to work on a project, a set of all possible
  outcomes for each situation is called a Sample
  space or probability space.
• Sample Space Domain of Random Variable

Will see later
4
Events

• An event E is any set of possible outcomes in S
• That is, E ? S
• E.g., the event that less than 50 people show up
  for our next class is represented as the set 1,
  2, , 49 of values of the variable V ( of
  people here next class).
• Can be anything from 1 up to 49

5
Probability

• The probability p PrE ? 0,1 of an event E
  is a real number representing our degree of
  certainty that E will occur.
• If PrE 1, then E is absolutely certain to
  occur,
• If PrE 0, then E is absolutely certain not to
  occur,
• If PrE ½, then we are maximally uncertain
  about whether E will occur that is,
• How do we interpret other values of p?

Note We could also define probabilities for more
general propositions, as well as events.
6
Four Definitions of Probability

• Several alternative definitions of probability
  are commonly encountered
• Frequentist, Bayesian, Laplacian, Axiomatic
• They have different strengths weaknesses,
  philosophically speaking.
• But fortunately, they coincide with each other
  and work well together, in the majority of cases
  that are typically encountered.

7
Probability Laplacian Definition

• First, assume that all individual outcomes in the
  sample space are equally likely to each other
• Then, the probability of any event E is given by,
  PrE E/S. Very simple!
• Problems Still needs a definition for equally
  likely, and depends on the existence of some
  finite sample space S in which all outcomes in S
  are, in fact, equally likely.

8
Example

• What is the probability that a five card poker
  hand contains exactly one ace?
• There are 4 ways to specify the ace.
• Once the ace is chosen there are C(48,4) ways to
  choose non-aces
• So 4C(48,4) hands with exactly one ace.
• Since there are C(52,5) equally likely hands
• Then 4C(48,4)/C(52,5) .30

9
Example

• What is the probability that a five card poker
  hand contains three aces and two jacks?
• (4,3) 4 ways to select three aces
• (4,2) 6 ways to select two jacks
• Then 6.4/(52,5) .000009234

10
Probability Distribution

• When there are n possible outcomes
• x1, x2, x3, xn and each outcome is assigned a
  probability p(s) such that the probability of
  each outcome is a nonnegative real number no
  greater than 1 and the sum of the probabilities
  of all possible outcomes is 1,
• THEN The function p from the set of all events
  of the sample space S is called a probability
  distribution.

11
Probabilities of Mutually Complementary Events

• Let E be an event in a sample space S.
• Then, E represents the complementary event,
  saying that the actual value of V?E.
• Theorem PrE 1 - PrE
• This can be proved using the Laplacian definition
  of probability, since PrE E/S
  (S-E)/S 1 - E/S 1 - PrE.
• Other definitions can also be used to prove it.

12
Probability vs. Odds

• You may have heard the term odds.
• It is widely used in the gambling community.
• This is not the same thing as probability!
• But, it is very closely related.
• The odds in favor of an event E means the
  relative probability of E compared with its
  complement E.
• O(E) Pr(E)/Pr(E).
• E.g., if p(E) 0.6 then p(E) 0.4 and O(E)
  0.6/0.4 1.5.
• Odds are conventionally written as a ratio of
  integers.
• E.g., 3/2 or 32 in above example. Three to two
  in favor.
• The odds against E just means 1/O(E).

13
Example 1 Balls-and-Urn

• Suppose an urn contains 4 blue balls and 5 red
  balls.
• An example experiment Shake up the urn, reach in
  (without looking) and pull out a ball.
• A random variable V Identity of the chosen
  ball.
• The sample space S The set ofall possible
  values of V
• In this case, S b1,,b9
• An event E The ball chosen isblue E
  ______________
• What are the odds in favor of E?
• What is the probability of E?
• (Use Laplacian defn.)

b1
b2
b9
b7
b5
b3
b8
b4
b6
14
Example 2 Seven on Two Dice

• Experiment Roll a pair offair (unweighted)
  6-sided dice.
• Describe a sample space for thisexperiment that
  fits the Laplacian definition.
• Using this sample space, represent an event E
  expressing that the upper spots sum to 7.
• What is the probability of E?

15
Probability of Unions of Events

• Let E1,E2 ? S
• Then we have Theorem PrE1? E2 PrE1
  PrE2 - PrE1?E2
• By the inclusion-exclusion principle, together
  with the Laplacian definition of probability.
• You should be able to easily flesh out the proof
  yourself at home.

16
Example

• What is the probability that a positive integer
  selected at random from the set of positive
  integers not exceeding 100 is divisible by either
  2 or 5?
• Let E1 be the event that integer is divisible by
  2
• Let E2 be the event that integer is divisible by
  5.
• Then E1 ? E2 is the event that is either 2 or 5
  and E1 n E2 is the event that is divisible by
  both 2 and 5 or equivalently that is divisible by
  10.
• p(E1 ? E2 ) p(E1) p(E2) p(E1 n E2)
• 50/100 20/100 10/100 3/5

17
Mutually Exclusive Events

• Two events E1, E2 are called mutually exclusive
  if they are disjoint E1?E2 ?
• Note that two mutually exclusive events cannot
  both occur in the same instance of a given
  experiment.
• For mutually exclusive events, PrE1 ? E2
  PrE1 PrE2.
• Follows from the sum rule of combinatorics.

18
Exhaustive Sets of Events

• A set E E1, E2, of events in the sample
  space S is called exhaustive iff
  .
• An exhaustive set E of events that are all
  mutually exclusive with each other has the
  property that

19
Independent Events

• Two events E,F are called independent if
  PrE?F PrEPrF.
• Relates to the product rule for the number of
  ways of doing two independent tasks.
• Example Flip a coin, and roll a die.
• Pr(coin shows heads) ? (die shows 1)
• Prcoin is heads Prdie is 1 ½1/6 1/12.

20
Conditional Probability

• Let E,F be any events such that PrFgt0.
• Then, the conditional probability of E given F,
  written PrEF, is defined as PrEF
  PrE?F/PrF.
• This is what our probability that E would turn
  out to occur should be, if we are given only the
  information that F occurs.
• If E and F are independent then PrEF PrE.
• ? PrEF PrE?F/PrF PrEPrF/PrF
  PrE

21
Conditional Probability

• There are two bears - white and dark.
• What is p(both male)
• S (ff, mf, fm, mm), E (mm), P(E) ¼
• What is the probability that both are male if you
  knew one is a male?
• E (mm), F One is male (mf, fm, mm)
• P(F) 3/4
• EnF (mm)
• P(EnF) 1/4
• P(E F) P(EnF) / P(F) ¼ / ¾ 1/3

22
Random Variable

• Random Variable (RV) is a function from the
  sample space of an experiment to the set of real
  numbers. That is, a random variable assigns a
  real number to each possible out come.
• Note RV is not a variable and it is not random.

23
Random Variable

• A random variable can be thought of as the
  numeric result of operating a non-deterministic
  mechanism or performing a non-deterministic
  experiment to generate a random result.
• Rolling a dice and recording the outcomes yields
  a random variable X with domain and range 1, 2,
  3, 4, 5, 6
• X (1) 1, X(2) 2, ., X(6)6

24
Random Variable Example

• If a fair coin is tossed four times, the sample
  space for this random experiment may be given as
• S HHHH,
• HHHT,HHTH,HTHH,THHH
• HHTT,HTHT,THHT,THTH,TTHH,
• HTTT,THTT,TTHT,TTTH,
• TTTT.
• For each of the 16 strings of Hs and Ts in S,
  we define the random variable X as X(x1x2x3x4)
  which counts the number of Hs that appear among
  the four components x1, x2, x3, x4

25
Random Variable Example

• X (HHHH) 4
• X (HHHT) X(HHTH) X(HTHH) X(THHH) 3
• X (HHTT) (HTHT) X(HTTH) X(THHT) X(THTH)
  X(TTHH) 2
• X (HTTT) X(THTT) X(TTHT) X(TTTH) 1
• X (TTTT) 0
• ? X associates each of the 16 strings of Hs and
  Ts in S with one of the nonnegative integers in
  0, 1, 2, 3, 4, i.e. X is a function with domain
  of S and codomain of R (real numbers).

26
Random Variable Example

• We can use the random variable X to express
  probability of certain events. For instance the
  event A results in two Hs and two Ts. The
  probability of A is probability of X2
• P(A) P(X 2) 6/16
• The probability distribution for this particular
  random variable X
• x P(X x)
• 0 1/16
• 1 4/16 1/4
• 2 6/16 3/8 P(X x) 0 for
  x ? 0, 1, 2, 3, 4.
• 3 4/16 1/4
• 4 1/16

27
6.4 Expected Value

• Since a random variable can be described by its
  probability distribution, it can be characterized
  by means of two measures its mean/expected
  value, which is a measure of central tendency,
  and its variance, which is a measure of
  dispersion.
• Average Time Complexity of an algorithm
• Expected value can be used

28
Expected Value

• The mean or expected value of X is defined as
• e.g. when a fair coin is tossed four times
  then

0 . 1/16 1 . 4/16 2 . 6/16 3 . 4/16 4 .
1/16
0412124
2
16
i.e. the expected value E(X) is found to be among
the values determined by the random variable X
2 with a probability of 3/8.
29
Variance of Random Variable

• Suppose X is the random variable defined on the
  sample space Sxa, b, c, where X(a) -1, X(b)
  0, X(c) 1 and P(Xx) 1/3, for x -1, 0, 1,
  then E(X) 0.
• Y is a random variable defined on the sample
  space Sy r, s, t, u, v, where Y(r) -4, Y(s)
  -2, Y(t) 0, Y(u) 2, Y(v) 4, and P(Y y)
  1/5, for y -4, -2, 0, 2, 4, we get the same
  mean that is E(Y) 0.
• However, the values determined by Y are more
  spread out about the mean of 0 than the values
  determined by X.
• This is measured by variance, denoted by s2x.

30
Variance of Random Variable

• s2x Var(x) E(X E(X))2 ?(x E(X))2 . P(X
  x)
• Suppose the probability distribution for X is
• x P(X x)
• 1 1/5 E(X) 17/5
• 3 2/5 s2x 66/25
• 4 1/5 and standard deviation of X
• 6 1/5. sx 1.62

31
Expected value and Variance

• Tossing a fair coin 4 times
• x P(X x) E(X) 2
• 0 1/16 and
• 1 4/16 1/4 sx 1
• 2 6/16 3/8
• 3 4/16 1/4
• 4 1/16

32
Probabilistic Method

• Probabilistic algorithms are algorithms that make
  random choices at one or more steps. Some times
  called randomize algorithms, the result and /or
  the way the result is obtained depends on chance.
• For example, simulating the behavior of some
  existing or planned system over time.

33
Probabilistic Method

• For some problems where trivial exhaustive search
  is not feasible probabilistic algorithms can be
  applied giving a result that is correct with a
  probability less than one (eg. primality testing,
  string equality testing). The probability of
  failure can be made arbitrary small by repeated
  applications of the algorithm.

34
7.1 Recurrence Relations

• A recurrence relation (R.R., or just recurrence)
  for a sequence an is an equation that expresses
  an in terms of one or more previous elements a0,
  , an-1 of the sequence, for all nn0.
• i.e., just a recursive definition, without the
  base cases.
• A particular sequence (described non-recursively)
  is said to solve the given recurrence relation if
  it is consistent with the definition of the
  recurrence.
• A given recurrence relation may have many
  solutions.

35
Recurrence Relation Example

• Consider the recurrence relation
• an 2an-1 - an-2 (n2).
• Suppose a0 3 and a1 5, what are a2, a3?
• a2 a1 - a0 5 3 2
• a3 a2 - a1 2 5 -3
• Which of the following are solutions? an
  3n an 2n
• an 5

Yes
3n 2. 3(n-1) - 3. (n-2)
No
Yes
36
Example Applications

• Recurrence relation for growth of a bank account
  with P interest per given period
• Mn Mn-1 (P/100)Mn-1
• Growth of a population in which each organism
  yields 1 new one every period starting 2 time
  periods after its birth.
• Pn Pn-1 Pn-2 (Fibonacci relation)

37
Solving Compound Interest RR

• Mn Mn-1 (P/100)Mn-1
• (1 P/100) Mn-1
• r Mn-1 (let r 1 P/100)
• r (r Mn-2)
• rr(r Mn-3) and so on to
• rn M0

38
Towers of Hanoi Example

• Problem Get all disks from peg 1 to peg 2.
• Rules (a) Only move 1 disk at a time.
• (b) Never set a larger disk on a smaller one.

Peg 1
Peg 2
Peg 3
39
Hanoi Recurrence Relation

• Let Hn moves for a stack of n disks.
• Here is the optimal strategy
• Move top n-1 disks to spare peg. (Hn-1 moves)
• Move bottom disk. (1 move)
• Move top n-1 to bottom disk. (Hn-1 moves)
• Note that Hn 2Hn-1 1
• The of moves is described by a Rec. Rel.
• Result 2n -1

40
A recursive solution

• Lets see a demo of it
• http//members.shaw.ca/orionx/th/Hanoi.html?Englis
  h
• You can implement a recursive function to solve
  this
• F(n) 2. F(n-1) 1
• If it takes 1 sec for each move and we have 64
  gold disks, according to the myth ?
• It will take 264 -1 18446744073709551615
• End of days!!! ? 500 billion years