Probability and Statistics with Reliability, Queuing and Computer Science Applications: Chapter 3

1
Probability and Statistics with Reliability,
Queuing and Computer Science Applications
Chapter 3

• Continuous Random Variables

2
Definitions

• Distribution function
• If FX(x) is a continuous function of x, then X is
  a continuous random variable.
• FX(x) discrete in x ? Discrete rvs
• FX(x) piecewise continuous ? Mixed rvs

3
Definitions (Continued)

• Equivalence
• CDF (cumulative distribution function)
• PDF (probability distribution function)
• Distribution function
• FX(x) or FX(t) or F(t)

4
Probability Density Function (pdf)

• X continuous rv, then,
• pdf properties

5
Definitions(Continued)

• Equivalence pdf
• probability density function
• density function
• density
• f(t)

For a non-negative random variable
6
Exponential Distribution

• Arises commonly in reliability queuing theory.
• A non-negative random variable
• It exhibits memoryless (Markov) property.
• Related to (the discrete) Poisson distribution
• Interarrival time between two IP packets (or
  voice calls)
• Time to failure, time to repair etc.
• Mathematically (CDF and pdf, respectively)

7
CDF of exponentially distributed random
variable with ? 0.0001
F(t)
12500 25000 37500 50000
t
8
Exponential Density Function (pdf)
f(t)
t
9
Memoryless property

• Assume X gt t. We have observed that the
  component has not failed until time t.
• Let Y X - t , the remaining (residual) lifetime
• The distribution of the remaining life, Y, does
  not depend on how long the component has been
  operating. Distribution of Y is identical to that
  of X.

10
Memoryless property

• Assume X gt t. We have observed that the
  component has not failed until time t.
• Let Y X - t , the remaining (residual) lifetime

11
Memoryless property (Continued)

• Thus Gt(y) is independent of t and is identical
  to the original exponential distribution of X.
• The distribution of the remaining life does not
  depend on how long the component has been
  operating.
• Its eventual breakdown is the result of some
  suddenly appearing failure, not of gradual
  deterioration.

12
Reliability as a Function of Time

• Reliability R(t) failure occurs after time t.
  Let X be the lifetime of a component subject to
  failures.
• Let N0 total no. of components (fixed) Ns(t)
  surviving ones Nf(t) failed one by time t.

13
Definitions (Continued)

• Equivalence
• Reliability
• Complementary distribution function
• Survivor function
• R(t) 1 -F(t)

14
Failure Rate or Hazard Rate

• Instantaneous failure rate h(t) (failures/10k
  hrs)
• Let the rv X be EXP( ?). Then,
• Using simple calculus the following apples to any
  rv,

15
Hazard Rate and the pdf

• h(t) ?t Conditional Prob. system will fail in
  
• (t, t ?t) given that it has survived
  until time t
• f(t) ?t Unconditional Prob. System will fail in
  
• (t, t ?t)
• Difference between
• probability that someone will die between 90 and
  91, given that he lives to 90
• probability that someone will die between 90 and
  91

16
Weibull Distribution

• Frequently used to model fatigue failure, ball
  bearing failure etc. (very long tails)
• Reliability
• Weibull distribution is capable of modeling DFR
  (a lt 1), CFR (a 1) and IFR (a gt1) behavior.
• a is called the shape parameter and ? is the
  scale parameter

17
Failure rate of the weibull distribution with
various values of ? and ? 1
5.0
1.0 2.0 3.0
4.0
18
Infant Mortality Effects in System Modeling

• Bathtub curves
• Early-life period
• Steady-state period
• Wear out period
• Failure rate models

19
Bathtub Curve

• Until now we assumed that failure rate of
  equipment is time (age) independent. In
  real-life, variation as per the bathtub shape has
  been observed

Failure Rate l(t)
Infant Mortality (Early Life Failures)
Wear out
Steady State
Operating Time
20
Early-life Period

• Also called infant mortality phase or reliability
  growth phase
• Caused by undetected hardware/software defects
  that are being fixed resulting in reliability
  growth
• Can cause significant prediction errors if
  steady-state failure rates are used
• Availability models can be constructed and solved
  to include this effect
• Weibull Model can be used

21
Steady-state Period

• Failure rate much lower than in early-life period
• Either constant (age independent) or slowly
  varying failure rate
• Failures caused by environmental shocks
• Arrival process of environmental shocks can be
  assumed to be a Poisson process
• Hence time between two shocks has the exponential
  distribution

22
Wear out Period

• Failure rate increases rapidly with age
• Properly qualified electronic hardware do not
  exhibit wear out failure during its intended
  service life (Motorola)
• Applicable for mechanical and other systems
• Weibull Failure Model can be used

23
Bathtub curve DFR phase Initial design, constant
bug fixes CFR phase Normal operational phase IFR
phase Aging behavior
h(t)
(burn-in-period)
(wear-out-phase)
CFR (useful life)
DFR
IFR
t
Increasing fail. rate
Decreasing failure rate
24
Failure Rate Models

• We use a truncated Weibull Model
• Infant mortality phase modeled by DFR Weibull and
  the steady-state phase by the exponential

7 6 5 4 3 2 1 0
Failure-Rate Multiplier
0
2,190
4,380
6,570
8,760
10,950
13,140
15,330
17,520
Operating Times (hrs)
25
Failure Rate Models (cont.)

• This model has the form
• where
• steady-state failure rate
• is the Weibull shape parameter
• Failure rate multiplier

26
Failure Rate Models (cont.)

• There are several ways to incorporate time
  dependent failure rates in availability models
• The easiest way is to approximate a continuous
  function by a decreasing step function

7 6 5 4 3 2 1 0
Failure-Rate Multiplier
2,190
4,380
6,570
10,950
13,140
15,330
17,520
8,760
0
Operating Times (hrs)
27
Failure Rate Models (cont.)

• Here the discrete failure-rate model is defined
  by

28
Uniform Random Variable

• All (pseudo) random generators generate random
  deviates of U(0,1) distribution that is, if you
  generate a large number of random variables and
  plot their empirical distribution function, it
  will approach this distribution in the limit.
• U(a,b) ? pdf constant over the (a,b) interval and
  CDF is the ramp function

29
Uniform density
30
Uniform distribution

• The distribution function is given by

0 , x lt a, F(x)
, a lt x lt b 1
, x gt b.

31
Uniform distribution (Continued)
32
HypoExponential

• HypoExp multiple Exp stages in series.
• 2-stage HypoExp denoted as HYPO(?1, ?2). The
  density, distribution and hazard rate function
  are
• HypoExp results in IFR 0 ? min(?1, ?2)
• Disk service time may be modeled as a 3-stage
  Hypoexponential as the overall time is the sum of
  the seek, the latency and the transfer time

33
HypoExponential used in software rejuvenation
models

• Preventive maintenance is useful only if failure
  rate is increasing

34
Erlang Distribution

• Special case of HypoExp All stages have same
  rate.
• X gt t Nt lt r (Nt no. of stresses applied
  in (0,t) and Nt is Possion (param ?t). This
  interpretation gives,

35
Erlang Distribution

• Is used to approximate the deterministic one
• since if you keep the same mean but increase
  the number of stages, the pdf approaches the
  delta function in the limit
• Can also be used to approximate the uniform
  distribution

36
probability density functions (pdf)
If we vary r keeping r/? constant, pdf of r-stage
Erlang approaches an impulse function at r/ ?.
37
cumulative distribution functions (cdf)
And the cdf approaches a step function at r/?. In
other words r-stage Erlang can approximate a
deterministic variable.
38
Comparison of probability density functions (pdf)
T 1
39
Comparison of cumulative distribution functions
(cdf)
T 1
40
Gamma Random Variable

• Gamma density function is,
• Gamma distribution can capture all three failure
  modes, viz. DFR, CFR and IFR.
• a 1 CFR
• a lt1 DFR
• a gt1 IFR
• Gamma with a ½ and ? n/2 is known as the
  chi-square random variable with n degrees of
  freedom

41
HyperExponential Distribution

• Hypo or Erlang ? Sequential Exp( ) stages.
• Alternate Exp( ) stages ? HyperExponential.
• CPU service time may be modeled as HyperExp
• In workload based software rejuvenation model we
  found the sojourn times in many workload states
  have this distribution

42
Log-logistic Distribution

• Log-logistic can model DFR, CFR and IFR failure
  models simultaneously, unlike previous ones.
• For, ? gt 1, the failure rate first increases with
  t (IFR) after momentarily leveling off (CFR), it
  decreases (DFR) with time. This is known as the
  inverse bath tub shape curve
• Use in modeling software reliability growth

43
Hazard rate comparison
44
Defective Distribution

• If
• Example
• This defect (also known as the mass at infinity)
  could represent the probability that the program
  will not terminate (1-c). Continuous part can
  model completion time of program.
• There can also be a mass at origin.
  

45
Pareto Random Variable

• Also known as the power law or long-tailed
  distribution
• Found to be useful in modeling
• CPU time consumed by a request
• Webfile sizes
• Number of data bytes in FTP bursts
• Thinking time of a Web browser

46
Gaussian (Normal) Distribution

• Bell shaped pdf intuitively pleasing!
• Central Limit Theorem mean of a large number of
  mutually independent rvs (having arbitrary
  distributions) starts following Normal
  distribution as n ?
• µ mean, s std. deviation, s2 variance (N(µ,
  s2))
• µ and s completely describe the statistics. This
  is significant in statistical estimation/signal
  processing/communication theory etc.

47
Normal Distribution (contd.)

• N(0,1) is called normalized Guassian.
• N(0,1) is symmetric i.e.
• f(x)f(-x)
• F(z) 1-F(z).
• Failure rate h(t) follows IFR behavior.
• Hence, N( ) is suitable for modeling long-term
  wear or aging related failure phenomena.

48
Functions of Random Variables

• Often, rvs need to be transformed/operated upon.
• Y F (X) so, what is the density of Y ?
• Example Y X2
• If X is N(0,1), then,
• Above Y is also known as the ?2 distribution
  (with 1-degree of freedom).

49
Functions of RVs (contd.)

• If X is uniformly distributed, then, Y
  -?-1ln(1-X) follows Exp(?) distribution
• transformations may be used to generate random
  variates (or deviates) with desired
  distributions.

50
Functions of RVs (contd.)

• Given,
• A monotone differentiable function,
• Above method suggests a way to get the random
  variates with desired distribution.
• Choose F to be F.
• Since, YF(X), FY(y) y and Y is U(0,1).
• To generate a random variate with X having
  desired distribution, generate U(0,1) random
  variable Y, then transform y to x F-1(y) .
• This inversion can be done in closed-form,
  graphically or using a table.

51
Jointly Distributed RVs

• Joint Distribution Function
• Independent rvs iff the following holds

52
Joint Distribution Properties

53
Joint Distribution Properties (contd)

54
Order statistics kofn, TMR
55
Order Statistics KofN

• X1 ,X2 ,..., Xn iid (independent and identically
  distributed) random variables with a common
  distribution function F().
• Let Y1 ,Y2 ,...,Yn be random variables obtained
  by permuting the set X1 ,X2 ,..., Xn so as to be
  in
• increasing order.
• To be specific
• Y1 minX1 ,X2 ,..., Xn and
• Yn maxX1 ,X2 ,..., Xn

56
Order Statistics KofN (Continued)

• The random variable Yk is called the k-th ORDER
  STATISTIC.
• If Xi is the lifetime of the i-th component in a
  system of n components. Then
• Y1 will be the overall series system lifetime.
• Yn will denote the lifetime of a parallel
  system.
• Yn-k1 will be the lifetime of an k-out-of-n
  system.

57
Order Statistics KofN (Continued)

• To derive the distribution function of Yk, we
  note that the
• probability that exactly j of the Xi's lie in (-?
  ,y and (n-j) lie in (y, ?) is

58
Applications of order statistics

• Reliability of a k out of n system
• Series system
• Parallel system
• Minimum of n EXP random variables is special case
  of Y1 minX1,,Xn where XiEXP(?i)
• Y1EXP(? ?i)
• This is not true (that is EXP dist.) for the
  parallel case

59
Triple Modular Redundancy (TMR)
R(t)
Voter
R(t)
R(t)

• An interesting case of order statistics occurs
  when we consider the Triple Modular Redundant
  (TMR) system (n 3 and k 2). Y2 then denotes
  the time until the second component fails. We get

60
TMR (Continued)

• Assuming that the reliability of a single
  component is given by,
• we get

61
TMR (Continued)

• In the following figure, we have plotted RTMR(t)
  vs t as well as R(t) vs t.

62
TMR (Continued)
63
(No Transcript)
64
(No Transcript)
65
(No Transcript)
66
(No Transcript)
67
(No Transcript)
68
(No Transcript)
69
(No Transcript)
70
(No Transcript)
71
(No Transcript)
72
(No Transcript)
73
(No Transcript)
74
Cold standby (dynamic redundancy)
X
Y

• Lifetime of
• Active
• EXP(?)

Lifetime of Spare EXP(?)
Total lifetime 2-Stage Erlang
Assumptions Detection Switching perfect spare
does not fail
75
Sum of RVs Standby Redundancy

• Two independent components, X and Y
• Series system (Zmin(X,Y))
• Parallel System (Zmax(X,Y))
• Cold standby the life time ZXY

76
Sum of Random Variables

• Z F(X, Y) ? ((X, Y) may not be independent)
• For the special case, Z X Y
• The resulting pdf is (assuming independence),
• Convolution integral (modify for the non-negative
  case)

77
Convolution (non-negative case)

• Z X Y, X Y are independent random variables
  (in this case, non-negative)
• The above integral is often called the
  convolution of fX and fY. Thus the density of the
  sum of two non-negative independent, continuous
  random variables is the convolution of the
  individual densities.

78
Cold standby derivation

• X and Y are both EXP(?) and independent.
• Then

79
Cold standby derivation (Continued)

• Z is two-stage Erlang Distributed

80
Convolution Erlang Distribution

• The general case of r-stage Erlang Distribution
• When r sequential phases have independent
  identical exponential distributions, then the
  resulting density is known as r-stage (or
  r-phase) Erlang and is given by

81
Convolution Erlang (Continued)
82
Warm standby

• With Warm spare, we have
• Active unit time-to-failure EXP(?)
• Spare unit time-to-failure EXP(?)
• 2-stage hypoexponential
  distribution

83
Warm standby derivation

• First event to occur is that either the active or
  the spare will fail. Time to this event is
  minEXP(?),EXP(?) which is EXP(? ?).
• Then due to the memoryless property of the
  exponential, remaining time is still EXP(?).
• Hence system lifetime has a two-stage
  hypoexponential distribution with parameters
• ?1 ? ? and ?2 ? .

84
Warm standby derivation (Continued)

• X is EXP(?1) and Y is EXP(?2) and are independent
• ?1 ?2
• Then fZ(t) is

85
Hot standby

• With hot spare, we have
• Active unit time-to-failure EXP(?)
• Spare unit time-to-failure EXP(?)
• 2-stage
  hypoexponential

86
TMR and TMR/simplexas hypoexponentials
TMR/Simplex
TMR
87
Hypoexponential general case

• Z , where X1 ,X2 , , Xr
• are mutually independent and Xi is exponentially
• distributed with parameter ?i (?i ?j for i
  j).
• Then Z is a r-stage hypoexponentially
• distributed random variable.

88
Hypoexponential general case
89
KofN system lifetime as a hypoexponential

• At least, k out of n units should be operational
  for the
• system to be Up.

EXP((n-1)?)
EXP((k-1)?)
EXP(k?)
EXP(n?)
EXP(?)
...
...
Yn-k2
Yn-k1
Y2
Yn
Y1
90
KofN with warm spares

• At least, k out of n s units should be
  operational
• for the system to be Up. Initially n units are
  active
• and s units are warm spares.

EXP(n? (s-1) ?)
EXP(n?)
EXP(n? ?)
EXP(n? s?)
EXP(k?)
...
...
91
Sum of Normal Random Variables

• X1, X2, .., Xk are normal iid rvs, then, the
  rv Z (X1 X2 ..Xk) is also normal
  with,
• X1, X2, .., Xk are normal. Then,
  
• follows Gamma or the ?2 (with
  n-degrees of freedom) distribution