Probability: the mathematics of Randomness

1
Probability the mathematics of Randomness

• Chapter 5 part IV Joint RVs

2
5.4 Joint Distributions and Independence

• Definition joint distribution is, roughly
  speaking, a generalization of a random variable
  extended to multiple random variables
• Example Roll two six-sided dice, and look at the
  side facing up. Let X be the number on the first
  die, and Y be the number on the second die.

3
5.4 Joint Distributions and Independence

• What is the (joint) probability of X 1 and Y
  2?
• PX1,Y2 PX1PY2 (by independence)
• 1/6 (1/6) 1/36
• What is the probability of getting a total of 7?
• PXY 7 2(PX1,Y6PX2,Y5PX
  3,Y4)
• since you can switch variables
• 2 (3/36) 1/6

4
5.4 Joint Distributions and Independence

• Note although the concepts discussed in this
  section generalize to multiple random variables,
  the following definitions and notation
  corresponds to cases involving only two random
  variables

5
5.4 Joint Distributions and Independence

• Jointly Discrete RVs
• Definition a joint probability function is a
  non-negative function, f(x,y), that gives the
  joint probability that both Xx and Yy
• f(x,y) PXx and Yy PXx, Yx
• For a discrete pair of RVs, f(x,y) is typically
  displayed in a table (called a Contingency Table)

6
5.4 Joint Distributions and Independence

• Example Let X be the outer diameter (in mm) of
  a bolt and Y be the inner diameter (in mm) of the
  corresponding nut. Use the info below to display
  f(x,y) in a table

7
5.4 Joint Distributions and Independence

• Use the table found above to find
• PXltY ( times XltY)/( observations) 8/9
• or add up probabilities from the table for
  cases which XltY
• PY-X3 PY28,X25PY27,X24PY26,X23
• 0 1/9 1/9 2/9
• PX25
• PX25,Y26PX25,Y27PX25,Y28
• 1/9 2/9 0 1/3

8
5.4 Joint Distributions and Independence

• Definition a marginal probability function is a
  probability for a single random variable obtained
  by summing the joint probability function over
  all possible values of the other variables
• To get the marginal probability function for X,
• i.e. sum over the columns of f(x,y)
• Similarly, and we would sum over the rows
  of f(x,y)

9
5.4 Joint Distributions and Independence

• Example contd Find the marginal probability
  functions for X and Y.

10
5.4 Joint Distributions and Independence

• Recall the conditional probability is defined as
  P(XxYy) P(Xx, Yy)/P(Yy)
• Definition the conditional probability function
  of X given Y is
• fXY(xy) f(x,y)/Sxf(x,y) f(x,y)/fy(y)
• and the conditional probability function of Y
  given X is
• fYX(yx) f(x,y)/Syf(x,y) f(x,y)/fx(x)
• Note the given part must always be specified

11
5.4 Joint Distributions and Independence

• Example Find the conditional probability that
  X23 given that Y26.
• Write out the conditional probability function
  of X given Y26.

12
5.4 Joint Distributions and Independence

• Example Calculate fyx(yX25)

13
5.4 Joint Distributions and Independence

• Definition discrete rvs X and Y are independent
  if their joint probability function f(x,y) is the
  product of their respective marginal probability
  functions for all x and y
• f(x,y) fx(x)fy(y)
• If this does not hold, even for one (x,y) pair,
  then the variables X and Y are dependent
• This definition is easily extended to more than
  two random variables eg f(x,y,z)
  fx(x)fy(y)fz(z)

14
5.4 Joint Distributions and Independence

• Example (variation) Now suppose that when a
  person chooses one bolt, then the person randomly
  chooses one nut (this implies independence).
  Under this scenario, use the marginal
  probabilities to calculate the joint
  probabilities in a table.

15
5.4 Joint Distributions and Independence

• Definition The random variables X1, X2, , Xn
  are said to be independent and identically
  distributed (iid) if they all have the same
  marginal distribution and are all independent of
  each other
• This is the concept we use when we consider
  trials within specific distributions

16
5.4 Joint Distributions and Independence

• Extensions to continuous random variables
• These can be made by replacing summations with
  Riemann integrations
• Recall, in continuous RVs, PXx 0 so we must
  now consider probability intervals

17
5.4 Joint Distributions and Independence

• For the marginal probability density function
• Conditional probability function of X given Yy

18
5.5 Functions of Several Random Variables

• Goal We want to be able to answer the following
  question
• Given the joint distribution of random variables
  X1, X2, , Xn, what is the distribution function
  fu(u) of the variable U g(X1, X2, , Xn)
• We focus on linear combinations of random
  variables
• g(X1, X2, , Xn) a0 a1X1anXn

19
5.5 Functions of Several Random Variables

• For some U g(X1, X2, , Xn) finding fu(u) is
  very difficult or impossible. In these cases,
  computer simulation can be used to approximate
  the distribution (see section 5.5.2, focus on
  Example 22)
• We wont be discussing these types of functions

20
5.5 Functions of Several Random Variables

• Example (variation on nuts and bolts)
• Let X be the outer diameter of a bolt (mm)
• Let Y be the inner diameter of a nut (mm)
• Assume a person randomly chooses one bolt and
  then randomly chooses one nut. Therefore, we can
  reasonably assume that these two random variables
  are independent.

21
5.5 Functions of Several Random Variables

• We have the following joint and marginal
  distributions for X and Y

22
5.5 Functions of Several Random Variables

• Suppose we are interested in U Y-X. Can we
  determine fu(u)?

23
5.5 Functions of Several Random Variables

• Mean and Variances for Linear Combinations of
  Random Variables
• Example Using the example, U Y-X, find EU.
• Can you think of a way to calculate E(U) without
  knowing fu(u)?

24
5.5 Functions of Several Random Variables

• Mean and Variances for Linear Combinations of
  Random Variables
• Proposition If X1, X2, , Xn are n independent
  random variables, and a0, a1, , an are n1 known
  constants, then the random variable U
  a0a1X1anXn has the following mean and
  variance
• E(U) a0a1 EX1a2 EX2an EXn
• Var(U) a12 VarX1a22 VarX2an2 VarXn
• Note if the RVs are not independent, the
  formulas for EU still holds but the variance
  does not

25
5.5 Functions of Several Random Variables

• Back to the example
• U Y-X (1)Y (-1)X
• EU (1) EY (-1) EX
• VarU (1)2 VarY (-1)2 VarX
• VarY VarX

26
5.5 Functions of Several Random Variables

• Example Below are probability functions for two
  independent random variables X and Y along with
  their means and variances.

27
5.5 Functions of Several Random Variables

• Evaluate the following
• E(3 2X 3Y) E(3) E(2X) E(3Y)
• 3 2EX 3EY
• 3 2(3.3) 3(25) -65.4
• Var(3 2X 3Y) Var(3) Var(2X) Var(-3Y)
• 0 22Var(X) (-3)2Vary(Y)
• 4(1.91) 9(65)
• 592.64

28
5.5 Functions of Several Random Variables

• Example Suppose XN(5,2), YN(10,4) and X and Y
  are independent.
• Note EX 5, EY 10, Var(X) 2, Var(Y) 4
• E( 1 X 2Y) 1 5 2(10) 14
• Var( 1 X 2Y) 2 22 (4) 18

29
5.5 Functions of Several Random Variables

• Example Let XBin(10,.5) and YPoi(3)
• Note EX 10(.5) 5 EY 3
• Var(X) 10(.5)(.5) 2.5 Var(Y) 3
• E(52X-7Y) 5 2(5) 7(3) -6
• Var(52X-7Y) 22(2.5) (-7)2(3) 10 147
  157

30
5.5 Functions of Several Random Variables

• A common function we are often interested in is
  the sample mean
• Note that this is a linear combination of random
  variables, just like U!

31
5.5 Functions of Several Random Variables

• Fact1 If X1, X2, , Xn are iid with EXi µ and
  VarXis2 for i1,2,,n, then

32
5.5 Functions of Several Random Variables

• Example Suppose X1, X2, , X10 are iid N(5,2)
• Compute the expectation and variance of the
  sample mean

33
5.5 Functions of Several Random Variables

• Fact2 If X1, X2, , Xn are iid N(µ,s2) then
• Example Suppose X1, X2, , X20 are iid N(-2,10)
  rvs. What is the distribution of

34
5.5 Functions of Several Random Variables

• Propagation of Error Formulas
• For when Ug(X1,X2,,Xn) is not a linear function
• If X1,X2,,Xn are independent rvs, for small
  enough variances, VarX1, VarX2,,VarXn, the rv U
  has an approximate mean
• EU g(EX1,EX2,,EXn)
• and approximate variance

35
5.5 Functions of Several Random Variables

• Example Suppose that X, Y and Z are independent
  rvs with
• EX 2 VarX2.5
• EY 6 VarY 3
• EZ 0.5 VarZ 0.88
• Find the expected value and variance of
• g YeZ X2.

36
5.5 Functions of Several Random Variables

• E(g) E(YeZ X2) g(EX,EY,EX)
• (EY) e(EZ) (EX)2
• (6) e(0.5) (2)2
• 13.892

37
5.5 Functions of Several Random Variables

• Var(g)

38
5.5 Functions of Several Random Variables

• Example 24 (Vardeman and Jobe) pg 311
• Below is a schematic of an assembly of three
  resistors. If R1, R2, and R3 are the respective
  resistances of the three resistors making up the
  assembly, standard theory says
• R the assembly resistance
• is related to R1, R2 and R3 by
• A large lot of resistors is manufactured and has
  a mean resistance of 100 ? with a standard
  deviation of 2 ?. If three resistors are taken
  at random from the lot, find the approximate mean
  and standard deviation for the resulting assembly
  resistance.

39
5.5 Functions of Several Random Variables

• E(R) R(ER1, ER2, ER3)

40
5.5 Functions of Several Random Variables

• Note
• Var(R)

41
5.5 Functions of Several Random Variables

• Central Limit Theorem
• Consider a data set that is uniform in shape and
  has possible values in the range 0 to 10 (the
  technical distribution is XUniform(0,10)

1/10
2
4
6
8
10
42
5.5 Functions of Several Random Variables

• The pdf for a uniform distribution is for
  XUnif(a,b), f(x)1/(b-a) (so for our data the
  pdf is 1/10). Consider randomly sampling 10
  observations from this distribution and taking
  the sample average of those 10 values.
• 1.23 8.39 8.34 5.37 1.99 3.94 9.73 5.06 1.39 0.70
• The sample average is 4.614
• And repeat 3.85 2.29 1.24 0.51 2.56 2.54 8.37
  3.26 0.90 4.47 with a sample average of 2.999

43
5.5 Functions of Several Random Variables

• Continue this process 25 times and look at a
  histogram of the sample means

44
5.5 Functions of Several Random Variables
45
5.5 Functions of Several Random Variables

• Repeat the same procedure with a continuous Gamma
  distribution

46
5.5 Functions of Several Random Variables
47
5.5 Functions of Several Random Variables
48
5.5 Functions of Several Random Variables

• The larger your sample size, the fewer
  repetitions it will take for the sample means to
  look normal

49
5.5 Functions of Several Random Variables

• Central Limit Theorem (CLT) If X1, X2, , Xn are
  iid rvs (with mean µ and variance s2), then for
  large n, the variable is approximately
  normally distributed.
• Recall we showed earlier has a mean µ and
  variance s2/n. By the CLT, when n is large
  (generally n 25) then

50
5.5 Functions of Several Random Variables

• Let X1, X2, , X50 be iid N(200, 100) rvs.
• What distribution does follow exactly?
• N ( 200, 100/50 2)
• Suppose X1, X2, , Xn are rvs with a mean of 5
  and a variance of 10.
• What distribution does follow? Is it exact
  or approximate?
• N (5, 10/n) which is approximate since
  the original data were not normally
  distributed.

51
5.5 Functions of Several Random Variables

• What is the probability of observing a sample
  mean that is greater than 4 in a sample of size
  n10?

52
5.5 Functions of Several Random Variables

• What is the probability of observing a sample
  mean that is between 4 and 6 in a sample size
  n10?

53
5.5 Functions of Several Random Variables

• What is the probability of observing a sample
  mean that is between 4 and 6 in a sample size
  n90?

54
5.5 Functions of Several Random Variables

• If you had taken a sample of 90 people and found
  that their sample mean was less than 4 or greater
  than 6, what might you conclude?
• There was a 99.74 chance of being between 4 and
  6
• Maybe I just got a REALLY rare sample of 90
  people
• -OR-
• Maybe the population mean I started with doesnt
  reflect the population like I thought it did
  (more likely scenario)

55
5.5 Functions of Several Random Variables

• Suppose X1, , X40 are iid Bin(12,0.6). Find the
  expected value and variance of the mean and the
  probability that the sample mean is less than 7.5.