6.2 Probability Theory Longin Jan Latecki Temple University

1
6.2 Probability TheoryLongin Jan LateckiTemple
University

• Slides for a Course Based on the TextDiscrete
  Mathematics Its Applications (6th Edition)
  Kenneth H. Rosen based on slides by
• Michael P. Frank and Andrew W. Moore

2
Terminology

• A (stochastic) experiment is a procedure that
  yields one of a given set of possible outcomes
• The sample space S of the experiment is the set
  of possible outcomes.
• An event is a subset of sample space.
• A random variable is a function that assigns a
  real value to each outcome of an experiment

Normally, a probability is related to an
experiment or a trial.
Lets take flipping a coin for example, what are
the possible outcomes?
Heads or tails (front or back side) of the coin
will be shown upwards.
After a sufficient number of tossing, we can
statistically conclude that the probability of
head is 0.5.
In rolling a dice, there are 6 outcomes. Suppose
we want to calculate the prob. of the event of
odd numbers of a dice. What is that probability?
3
Random Variables

• A random variable V is any variable whose value
  is unknown, or whose value depends on the precise
  situation.
• E.g., the number of students in class today
• Whether it will rain tonight (Boolean variable)
• The proposition Vvi may have an uncertain truth
  value, and may be assigned a probability.

4
Example 10

• A fair coin is flipped 3 times. Let S be the
  sample space of 8 possible outcomes, and let X be
  a random variable that assignees to an outcome
  the number of heads in this outcome.
• Random variable X is a function XS ? X(S),
  where X(S)0, 1, 2, 3 is the range of X, which
  is the number of heads, andS (TTT), (TTH),
  (THH), (HTT), (HHT), (HHH), (THT), (HTH)
• X(TTT) 0 X(TTH) X(HTT) X(THT) 1X(HHT)
  X(THH) X(HTH) 2X(HHH) 3
• The probability distribution (pdf) of random
  variable X is given by P(X3) 1/8, P(X2)
  3/8, P(X1) 3/8, P(X0) 1/8.

5
Experiments Sample Spaces

• A (stochastic) experiment is any process by which
  a given random variable V gets assigned some
  particular value, and where this value is not
  necessarily known in advance.
• We call it the actual value of the variable, as
  determined by that particular experiment.
• The sample space S of the experiment is justthe
  domain of the random variable, S domV.
• The outcome of the experiment is the specific
  value vi of the random variable that is selected.

6
Events

• An event E is any set of possible outcomes in S
• That is, E ? S domV.
• E.g., the event that less than 50 people show up
  for our next class is represented as the set 1,
  2, , 49 of values of the variable V ( of
  people here next class).
• We say that event E occurs when the actual value
  of V is in E, which may be written V?E.
• Note that V?E denotes the proposition (of
  uncertain truth) asserting that the actual
  outcome (value of V) will be one of the outcomes
  in the set E.

7
Probabilities

• We write P(A) as the fraction of possible worlds
  in which A is true
• We could at this point spend 2 hours on the
  philosophy of this.
• But we wont.

8
Visualizing A


Event space of all possible worlds
P(A) Area of reddish oval
Worlds in which A is true
Its area is 1
Worlds in which A is False
9
Probability

• The probability p PrE ? 0,1 of an event E
  is a real number representing our degree of
  certainty that E will occur.
• If PrE 1, then E is absolutely certain to
  occur,
• thus V?E has the truth value True.
• If PrE 0, then E is absolutely certain not to
  occur,
• thus V?E has the truth value False.
• If PrE ½, then we are maximally uncertain
  about whether E will occur that is,
• V?E and V?E are considered equally likely.
• How do we interpret other values of p?

Note We could also define probabilities for more
general propositions, as well as events.
10
Four Definitions of Probability

• Several alternative definitions of probability
  are commonly encountered
• Frequentist, Bayesian, Laplacian, Axiomatic
• They have different strengths weaknesses,
  philosophically speaking.
• But fortunately, they coincide with each other
  and work well together, in the majority of cases
  that are typically encountered.

11
Probability Frequentist Definition

• The probability of an event E is the limit, as
  n?8, of the fraction of times that we find V?E
  over the course of n independent repetitions of
  (different instances of) the same experiment.
• Some problems with this definition
• It is only well-defined for experiments that can
  be independently repeated, infinitely many times!
  
• or at least, if the experiment can be repeated in
  principle, e.g., over some hypothetical ensemble
  of (say) alternate universes.
• It can never be measured exactly in finite time!
• Advantage Its an objective, mathematical
  definition.

12
Probability Bayesian Definition

• Suppose a rational, profit-maximizing entity R is
  offered a choice between two rewards
• Winning 1 if and only if the event E actually
  occurs.
• Receiving p dollars (where p?0,1)
  unconditionally.
• If R can honestly state that he is completely
  indifferent between these two rewards, then we
  say that Rs probability for E is p, that is,
  PrRE p.
• Problem Its a subjective definition depends on
  the reasoner R, and his knowledge, beliefs,
  rationality.
• The version above additionally assumes that the
  utility of money is linear.
• This assumption can be avoided by using utils
  (utility units) instead of dollars.

13
Probability Laplacian Definition

• First, assume that all individual outcomes in the
  sample space are equally likely to each other
• Note that this term still needs an operational
  definition!
• Then, the probability of any event E is given by,
  PrE E/S. Very simple!
• Problems Still needs a definition for equally
  likely, and depends on the existence of some
  finite sample space S in which all outcomes in S
  are, in fact, equally likely.

14
Probability Axiomatic Definition

• Let p be any total function pS?0,1 such
  that ?s p(s) 1.
• Such a p is called a probability distribution.
• Then, the probability under p of any event E?S
  is just
• Advantage Totally mathematically well-defined!
• This definition can even be extended to apply to
  infinite sample spaces, by changing ???, and
  calling p a probability density function or a
  probability measure.
• Problem Leaves operational meaning unspecified.

15
The Axioms of Probability

• 0 lt P(A) lt 1
• P(True) 1
• P(False) 0
• P(A or B) P(A) P(B) - P(A and B)

16
Interpreting the axioms

• 0 lt P(A) lt 1
• P(True) 1
• P(False) 0
• P(A or B) P(A) P(B) - P(A and B)

The area of A cant get any smaller than 0
And a zero area would mean no world could ever
have A true
17
Interpreting the axioms

• 0 lt P(A) lt 1
• P(True) 1
• P(False) 0
• P(A or B) P(A) P(B) - P(A and B)

The area of A cant get any bigger than 1
And an area of 1 would mean all worlds will have
A true
18
Interpreting the axioms

• 0 lt P(A) lt 1
• P(True) 1
• P(False) 0
• P(A or B) P(A) P(B) - P(A and B)

19
These Axioms are Not to be Trifled With

• There have been attempts to do different
  methodologies for uncertainty
• Fuzzy Logic
• Three-valued logic
• Dempster-Shafer
• Non-monotonic reasoning
• But the axioms of probability are the only system
  with this property
• If you gamble using them you cant be
  unfairly exploited by an opponent using some
  other system di Finetti 1931

20
Theorems from the Axioms

• 0 lt P(A) lt 1, P(True) 1, P(False) 0
• P(A or B) P(A) P(B) - P(A and B)
• From these we can prove
• P(not A) P(A) 1-P(A)
• How?

21
Another important theorem

• 0 lt P(A) lt 1, P(True) 1, P(False) 0
• P(A or B) P(A) P(B) - P(A and B)
• From these we can prove
• P(A) P(A B) P(A B)
• How?

22
Probability of an event E

• The probability of an event E is the sum of the
  probabilities of the outcomes in E. That is
• Note that, if there are n outcomes in the event
  E, that is, if E a1,a2,,an then

23
Example

• What is the probability that, if we flip a coin
  three times, that we get an odd number of tails?
• (TTT), (TTH), (THH), (HTT), (HHT), (HHH), (THT),
  (HTH)
• Each outcome has probability 1/8,
• p(odd number of tails) 1/81/81/81/8 ½

24
Visualizing Sample Space

• 1. Listing
• S Head, Tail
• 2. Venn Diagram
• 3. Contingency Table
• 4. Decision Tree Diagram

25
Venn Diagram
Experiment Toss 2 Coins. Note Faces.
Tail
Event
TH
HT
HH
Outcome
TT
S
Sample Space
S HH, HT, TH, TT
26
Contingency Table
Experiment Toss 2 Coins. Note Faces.
nd
2
Coin
st
1
Coin
Head
Tail
Total
Outcome
SimpleEvent (Head on1st Coin)
Head
HH
HT
HH, HT
Tail
TH
TT
TH, TT
Total
HH,
TH
HT,
TT
S


S HH, HT, TH, TT
Sample Space
27
Tree Diagram
Experiment Toss 2 Coins. Note Faces.
H
HH
H
T
HT
Outcome
H
TH
T
T
TT
S HH, HT, TH, TT
Sample Space
28
Discrete Random Variable

• Possible values (outcomes) are discrete
• E.g., natural number (0, 1, 2, 3 etc.)
• Obtained by Counting
• Usually Finite Number of Values
• But could be infinite (must be countable)

29
Discrete Probability Distribution ( also called
probability mass function (pmf) )

• 1. List of All possible x, p(x) pairs
• x Value of Random Variable (Outcome)
• p(x) Probability Associated with Value
• 2. Mutually Exclusive (No Overlap)
• 3. Collectively Exhaustive (Nothing Left Out)
• 4. 0 ? p(x) ? 1
• 5. ? p(x) 1

30
Visualizing Discrete Probability Distributions
Table
Listing
Tails
f(x
)
p(x
)

• (0, .25), (1, .50), (2, .25)

Count
0
1
.25
1
2
.50
2
1
.25
p(x)
Graph
Equation
.50
n
!
x
n
x
?
p
x
p
p
(
)
(
)
?
?
1
.25
x
n
x
!
(
)
!
?
x
.00
0
1
2
31
Arity of Random Variables

• Suppose A can take on more than 2 values
• A is a random variable with arity k if it can
  take on exactly one value out of v1,v2, .. vk
• Thus

32
Mutually Exclusive Events

• Two events E1, E2 are called mutually exclusive
  if they are disjoint E1?E2 ?
• Note that two mutually exclusive events cannot
  both occur in the same instance of a given
  experiment.
• For mutually exclusive events, PrE1 ? E2
  PrE1 PrE2.

33
Exhaustive Sets of Events

• A set E E1, E2, of events in the sample
  space S is called exhaustive iff
  .
• An exhaustive set E of events that are all
  mutually exclusive with each other has the
  property that

34
An easy fact about Multivalued Random Variables

• Using the axioms of probability
• 0 lt P(A) lt 1, P(True) 1, P(False) 0
• P(A or B) P(A) P(B) - P(A and B)
• And assuming that A obeys

• Its easy to prove that

• And thus we can prove

35
Another fact about Multivalued Random Variables

• Using the axioms of probability
• 0 lt P(A) lt 1, P(True) 1, P(False) 0
• P(A or B) P(A) P(B) - P(A and B)
• And assuming that A obeys

• Its easy to prove that

36
Elementary Probability Rules

• P(A) P(A) 1
• P(B) P(B A) P(B A)

37
Bernoulli Trials

• Each performance of an experiment with only two
  possible outcomes is called a Bernoulli trial.
• In general, a possible outcome of a Bernoulli
  trial is called a success or a failure.
• If p is the probability of a success and q is the
  probability of a failure, then pq1.

38
Example

• A coin is biased so that the probability of heads
  is 2/3. What is the probability that exactly
  four heads come up when the coin is flipped
  seven times, assuming that the flips are
  independent?
• The number of ways that we can get four heads is
  C(7,4) 7!/4!3! 75 35
• The probability of getting four heads and three
  tails is (2/3)4(1/3)3 16/37
• p(4 heads and 3 tails) is C(7,4) (2/3)4(1/3)3
  3516/37 560/2187

39
Probability of k successes in n independent
Bernoulli trials.

• The probability of k successes in n independent
  Bernoulli trials, with probability of success p
  and probability of failure q 1-p is
  C(n,k)pkqn-k

40
Find each of the following probabilities when n
independent Bernoulli trials are carried out with
probability of success, p.

• Probability of no successes.
• C(n,0)p0qn-k 1(p0)(1-p)n (1-p)n
• Probability of at least one success.
• 1 - (1-p)n (why?)

41
Find each of the following probabilities when n
independent Bernoulli trials are carried out with
probability of success, p.

• Probability of at most one success.
• Means there can be no successes or one success
• C(n,0)p0qn-0 C(n,1)p1qn-1
• (1-p)n np(1-p)n-1
• Probability of at least two successes.
• 1 - (1-p)n - np(1-p)n-1

42
A coin is flipped until it comes ups tails. The
probability the coin comes up tails is p.

• What is the probability that the experiment ends
  after n flips, that is, the outcome consists of
  n-1 heads and a tail?
• (1-p)n-1p

43
Probability vs. Odds
ExerciseExpress theprobabilityp as a
functionof the odds in favor O.

• You may have heard the term odds.
• It is widely used in the gambling community.
• This is not the same thing as probability!
• But, it is very closely related.
• The odds in favor of an event E means the
  relative probability of E compared with its
  complement E. O(E) Pr(E)/Pr(E).
• E.g., if p(E) 0.6 then p(E) 0.4 and O(E)
  0.6/0.4 1.5.
• Odds are conventionally written as a ratio of
  integers.
• E.g., 3/2 or 32 in above example. Three to two
  in favor.
• The odds against E just means 1/O(E). 2 to 3
  against

44
Example 1 Balls-and-Urn

• Suppose an urn contains 4 blue balls and 5 red
  balls.
• An example experiment Shake up the urn, reach in
  (without looking) and pull out a ball.
• A random variable V Identity of the chosen
  ball.
• The sample space S The set ofall possible
  values of V
• In this case, S b1,,b9
• An event E The ball chosen isblue E
  ______________
• What are the odds in favor of E?
• What is the probability of E?

b1
b2
b9
b7
b5
b3
b8
b4
b6
45
Independent Events

• Two events E,F are called independent if
  PrE?F PrEPrF.
• Relates to the product rule for the number of
  ways of doing two independent tasks.
• Example Flip a coin, and roll a die.
• Pr(coin shows heads) ? (die shows 1)
• Prcoin is heads Prdie is 1 ½1/6 1/12.

46
Example
Suppose a red die and a blue die are rolled. The
sample space
Are the events sum is 7 and the blue die is 3
independent?
47
The events sum is 7 and the blue die is 3 are
independent
S 36
p(sum is 7 and blue die is 3) 1/36 p(sum is 7)
p(blue die is 3) 6/366/361/36 Thus, p((sum is
7) and (blue die is 3)) p(sum is 7) p(blue die
is 3)
48
Conditional Probability

• Let E,F be any events such that PrFgt0.
• Then, the conditional probability of E given F,
  written PrEF, is defined as PrEF
  PrE?F/PrF.
• This is what our probability that E would turn
  out to occur should be, if we are given only the
  information that F occurs.
• If E and F are independent then PrEF PrE.
• ? PrEF PrE?F/PrF PrEPrF/PrF
  PrE

49
Visualizing Conditional Probability

• If we are given that event F occurs, then
• Our attention gets restricted to the subspace F.
• Our posterior probability for E (after seeing F)
  correspondsto the fraction of F where Eoccurs
  also.
• Thus, p'(E)p(EnF)/p(F).

Entire sample space S
Event F
Event E
EventEnF
50
Conditional Probability Example

• Suppose I choose a single letter out of the
  26-letter English alphabet, totally at random.
• Use the Laplacian assumption on the sample space
  a,b,..,z.
• What is the (prior) probabilitythat the letter
  is a vowel?
• PrVowel __ / __ .
• Now, suppose I tell you that the letter chosen
  happened to be in the first 9 letters of the
  alphabet.
• Now, what is the conditional (orposterior)
  probability that the letteris a vowel, given
  this information?
• PrVowel First9 ___ / ___ .

1st 9letters
vowels
w
z
r
k
b
c
a
t
y
u
d
f
e
x
g
i
o
l
s
h
j
n
p
m
q
v
Sample Space S
51
Example

• What is the probability that, if we flip a coin
  three times, that we get an odd number of tails
  (event E), if we know that the event F, the
  first flip comes up tails occurs?
• (TTT), (TTH), (THH), (HTT), (HHT),
  (HHH), (THT), (HTH)
• Each outcome has probability 1/4,
• p(E F) 1/41/4 ½, where Eodd number of
  tails
• or p(EF) p(E?F)/p(F) 2/4 ½
• For comparison p(E) 4/8 ½
• E and F are independent, since p(E F) Pr(E).

52
Prior and Posterior Probability

• Suppose that, before you are given any
  information about the outcome of an experiment,
  your personal probability for an event E to occur
  is p(E) PrE.
• The probability of E in your original probability
  distribution p is called the prior probability of
  E.
• This is its probability prior to obtaining any
  information about the outcome.
• Now, suppose someone tells you that some event F
  (which may overlap with E) actually occurred in
  the experiment.
• Then, you should update your personal probability
  for event E to occur, to become p'(E) PrEF
  p(EnF)/p(F).
• The conditional probability of E, given F.
• The probability of E in your new probability
  distribution p' is called the posterior
  probability of E.
• This is its probability after learning that event
  F occurred.
• After seeing F, the posterior distribution p' is
  defined by letting p'(v) p(vnF)/p(F) for
  each individual outcome v?S.

53
6.3 Bayes Theorem Longin Jan LateckiTemple
University

• Slides for a Course Based on the TextDiscrete
  Mathematics Its Applications (6th Edition)
  Kenneth H. Rosen based on slides by
• Michael P. Frank and Wolfram Burgard

54
Bayes Rule

• One way to compute the probability that a
  hypothesis H is correct, given some data D
• This follows directly from the definition of
  conditional probability! (Exercise Prove it.)
• This rule is the foundation of Bayesian methods
  for probabilistic reasoning, which are very
  powerful, and widely used in artificial
  intelligence applications
• For data mining, automated diagnosis, pattern
  recognition, statistical modeling, even
  evaluating scientific hypotheses!

Rev. Thomas Bayes1702-1761
55
Bayes Theorem

• Allows one to compute the probability that a
  hypothesis H is correct, given data D

Set of Hj is exhaustive
56
Example 1 Two boxes with balls

• Two boxes first 2 blue and 7 red balls second
  4 blue and 3 red balls
• Bob selects a ball by first choosing one of the
  two boxes, and then one ball from this box.
• If Bob has selected a red ball, what is the
  probability that he selected a ball from the
  first box.
• An event E Bob has chosen a red ball.
• An event F Bob has chosen a ball from the first
  box.
• We want to find p(F E)

57
Example 2

• Suppose 1 of population has AIDS
• Prob. that the positive result is right 95
• Prob. that the negative result is right 90
• What is the probability that someone who has the
  positive result is actually an AIDS patient?
• H event that a person has AIDS
• D event of positive result
• PDH 0.95 PD ?H 1- 0.9
• PD PDHPHPD?HP?H
• 0.950.010.10.990.1085
• PHD 0.950.01/0.10850.0876

58
Whats behind door number three?

• The Monty Hall problem paradox
• Consider a game show where a prize (a car) is
  behind one of three doors
• The other two doors do not have prizes (goats
  instead)
• After picking one of the doors, the host (Monty
  Hall) opens a different door to show you that the
  door he opened is not the prize
• Do you change your decision?
• Your initial probability to win (i.e. pick the
  right door) is 1/3
• What is your chance of winning if you change your
  choice after Monty opens a wrong door?
• After Monty opens a wrong door, if you change
  your choice, your chance of winning is 2/3
• Thus, your chance of winning doubles if you
  change
• Huh?

59
(No Transcript)
60
Monty Hall Problem
Ci - The car is behind Door i, for i equal to 1,
2 or 3. Hij - The host opens Door j after the
player has picked Door i, for i and j equal to
1, 2 or 3. Without loss of generality, assume,
by re-numbering the doors if necessary, that the
player picks Door 1, and that the host then
opens Door 3, revealing a goat. In other words,
the host makes proposition H13 true. Then the
posterior probability of winning by not switching
doors is P(C1H13).
61
P(H13 C1 ) 0.5, since the host will always
open a door that has no car behind it, chosen
from among the two not picked by the player
(which are 2 and 3 here)
62
The probability of winning by switching is
P(C2H13), since under our assumption switching
means switching the selection to Door 2, since
P(C3H13) 0 (the host will never open the door
with the car)
The posterior probability of winning by not
switching doors is P(C1H13) 1/3.
63
Exercises 6, p. 424, and 16, p. 425
64
(No Transcript)
65
(No Transcript)
66
(No Transcript)
67
Continuous random variable
68
Continuous Prob. Density Function

• 1. Mathematical Formula
• 2. Shows All Values, x, and Frequencies, f(x)
• f(x) Is Not Probability
• 3. Properties

(Value, Frequency)
f(x)
?
f
x
dx
(
)
?
1
x
a
b
All x
(Area Under Curve)
Value
f
x
(
)
a
x
b
?
?
?
0,
69
Continuous Random Variable Probability
d
?
P
c
x
d
f
x
dx
(
)
(
)
?
?
?
c
f(x)
Probability Is Area Under Curve!
X
c
d
70
Probability mass function
In probability theory, a probability mass
function (pmf) is a function that gives the
probability that a discrete random variable is
exactly equal to some value. A pmf differs from
a probability density function (pdf) in that the
values of a pdf, defined only for continuous
random variables, are not probabilities as such.
Instead, the integral of a pdf over a range of
possible values (a, b gives the probability of
the random variable falling within that range.
Example graphs of a pmfs. All the values of a pmf
must be non-negative and sum up to 1. (right)
The pmf of a fair die. (All the numbers on the
die have an equal chance of appearing on top
when the die is rolled.)
71
Suppose that X is a discrete random variable,
taking values on some countable sample space  S
? R. Then the probability mass function  fX(x) 
for X is given by thus Note that this
explicitly defines  fX(x)  for all real numbers,
including all values in R that X could never
take indeed, it assigns such values a
probability of zero. Example. Suppose that X is
the outcome of a single coin toss, assigning 0
to tails and 1 to heads. The probability that X
x is 0.5 on the state space 0, 1 (this is a
Bernoulli random variable), and hence the
probability mass function is
72
Uniform Distribution

• 1. Equally Likely Outcomes
• 2. Probability Density
• 3. Mean Standard Deviation

f(x)
x
d
c
Mean Median
73
Uniform Distribution Example

• Youre production manager of a soft drink
  bottling company. You believe that when a
  machine is set to dispense 12 oz., it really
  dispenses 11.5 to 12.5 oz. inclusive.
• Suppose the amount dispensed has a uniform
  distribution.
• What is the probability that less than 11.8 oz.
  is dispensed?

74
Uniform Distribution Solution
f(x)
1.0
x
11.5
12.5
11.8

• P(11.5 ? x ? 11.8) (Base)(Height)
• (11.8 - 11.5)(1) 0.30

75
Normal Distribution

• 1. Describes Many Random Processes or Continuous
  Phenomena
• 2. Can Be Used to Approximate Discrete
  Probability Distributions
• Example Binomial
• Basis for Classical Statistical Inference
• A.k.a. Gaussian distribution

76
Normal Distribution

• 1. Bell-Shaped Symmetrical
• 2. Mean, Median, Mode Are Equal
• 4. Random Variable Has Infinite Range

Mean
light-tailed distribution
77
Probability Density Function

• f(x) Frequency of Random Variable x
• ? Population Standard Deviation
• ? 3.14159 e 2.71828
• x Value of Random Variable (-?lt x lt ?)
• ? Population Mean

78
Effect of Varying Parameters (? ?)
79
Normal Distribution Probability
Probability is area under curve!
80
Infinite Number of Tables
Normal distributions differ by mean standard
deviation.
Each distribution would require its own table.
Thats an infinite number!
81
Standardize theNormal Distribution
Normal Distribution
Standardized Normal Distribution
One table!
82
Intuitions on Standardizing

• Subtracting ? from each value X just moves the
  curve around, so values are centered on 0 instead
  of on ?
• Once the curve is centered, dividing each value
  by ?gt1 moves all values toward 0, pressing the
  curve

83
Standardizing Example
Normal Distribution
84
Standardizing Example
Normal Distribution
Standardized Normal Distribution
85
(No Transcript)
86
6.4 Expected Value and VarianceLongin Jan
LateckiTemple University

• Slides for a Course Based on the TextDiscrete
  Mathematics Its Applications (6th Edition)
  Kenneth H. Rosen based on slides by Michael P.
  Frank

87
Expected Values

• For any random variable V having a numeric
  domain, its expectation value or expected value
  or weighted average value or (arithmetic) mean
  value ExV, under the probability distribution
  Prv p(v), is defined as
• The term expected value is very widely used for
  this.
• But this term is somewhat misleading, since the
  expected value might itself be totally
  unexpected, or even impossible!
• E.g., if p(0)0.5 p(2)0.5, then ExV1, even
  though p(1)0 and so we know that V?1!
• Or, if p(0)0.5 p(1)0.5, then ExV0.5 even
  if V is an integer variable!

88
Derived Random Variables

• Let S be a sample space over values of a random
  variable V (representing possible outcomes).
• Then, any function f over S can also be
  considered to be a random variable (whose actual
  value f(V) is derived from the actual value of
  V).
• If the range R rangef of f is numeric, then
  the mean value Exf of f can still be defined,
  as

89
Recall that a random variable X is actually a
function f S ? X(S), where S is the sample
space and X(S) is the range of X. This fact
implies that the expected value of X is
Example 1. Expected Value of a Die. Let X be the
number that comes up when a die is rolled.
90
Example 2

• A fair coin is flipped 3 times. Let S be the
  sample space of 8 possible outcomes, and let X be
  a random variable that assignees to an outcome
  the number of heads in this outcome.
• E(X) 1/8X(TTT) X(TTH) X(THH) X(HTT)
  X(HHT) X(HHH) X(THT) X(HTH) 1/80 1
  2 1 2 3 1 2 12/8 3/2

91
Linearity of Expectation Values

• Let X1, X2 be any two random variables derived
  from the same sample space S, and subject to the
  same underlying distribution.
• Then we have the following theorems
• ExX1X2 ExX1 ExX2
• ExaX1 b aExX1 b
• You should be able to easily prove these for
  yourself at home.

92
Variance Standard Deviation

• The variance VarX s2(X) of a random variable
  X is the expected value of the square of the
  difference between the value of X and its
  expectation value ExX
• The standard deviation or root-mean-square (RMS)
  difference of X is s(X) VarX1/2.

93
Example 15

• What is the variance of the random variable X,
  where X is the number that comes up when a die is
  rolled?
• V(X) E(X2) E(X)2
• E(X2) 1/612 22 32 42 52 62 91/6
• V(X) 91/6 (7/2) 2 35/12 2.92