Some standard univariate probability distributions

1
Some standard univariate probability distributions

• Characteristic function, moment generating
  function, cumulant generating functions
• Discrete distribution
• Continuous distributions
• Some distributions associated with normal
• References

2
Characteristic function, moment generating
function, cumulant generating functions

• Characteristic function is defined as expectation
  of the function - e(itx)
• Moment generating function is defined as
  (expectation of e(tx))
• Moments can be calculated in the following way.
  Obtain derivative of M(t) and take value of it at
  t0
• Cumulant generting function is defined as
  logarithm of characteristic function

3
Discrete distributions Binomial

• Let us assume that we carry experiment and result
  of the experiment can be success or failure.
  Probability of success is p. Then probability
  of failure will be q1-p. We carry experiments n
  times. What is probability of k successes
• Characteristic function
• Moment generating function
• Find first and second moments

4
Discrete distributions Poisson

• When number of trials (n) is large and
  probability of successes (p) is small and np is
  finite and tends to ? then binomial distribution
  converges to Poisson distribution
• Poisson distribution can be expected to describe
  the distribution an event that occurs rarely in a
  short period. It is used in counting statistics
  to describe of number of registered photons.
• Find characteristic and moment generating
  functions.
• Characteristic function is
• What is the first moment?

5
Discrete distributions Negative Binomial

• Consider experiment Probability of success is
  p and probability of failure q1-p. We carry out
  experiment until k-th success. We want to find
  probability of j failures. (It is called
  sequential sampling. Sampling is carried out
  until stopping rule is satisfied). If we have j
  failure then it means that we number of trials is
  kj. Last trial was success. Then probability
  that we will have j failures is
• It is called negative binomial because
  coefficients are from negative binomial series
  p-k(1-q)-k
• Characteristic function is
• What is the moment generating function? What is
  the first moment?

6
Continuous distributions uniform

• Simplest form of continuous distribution is the
  uniform with density
• Distribution is
• Moments and other properties are calculated
  easily.

7
Continuous distributions exponential

• Density of exponential distribution has the form
• This distribution has two origins.
• Maximum entropy. If we know that random variable
  is non-negative and we know its first moment
  1/? then maximum entropy distribution has the
  exponential form.
• From Poisson type random processes. If
  probability distribution of j(t) events occurring
  during time interval 0t) is a Poisson with mean
  value ? t then probability of time elapsing till
  first event occurs has the exponential
  distribution. Let Tr denotes time elapsed until
  r-th event
• Putting r1 we get e(- ?t). Taking into account
  that P(T1gtt) 1-F1(t) and getting its derivative
  wrt t we arrive to exponential distribution
• This distribution together with Poisson is widely
  used in reliability studies, life testing etc.

8
Continuous distributions Gamma

• Gamma distribution can be considered as
  generalisation of exponential distribution. It
  has the form
• It is probability of time t elapsing befor r
  events happens
• Characteristic function of this distribution is

9
Continuous distributions Normal

• Perhaps the most popular and widely used
  continuous distribution is the normal
  distribution. Main reason for this is that that
  usually random variable is the sum of the many
  random variables. According to central limit
  theorem under some conditions (for example
  random variables are independent. first and
  second and third moments exist and finite then
  distribution of sum of random variables converges
  to normal distribution)
• Density of the normal distribution has the form
• Another remarkable fact is that if we know mean
  value and variance only then random variable has
  the normal distribution.
• There many tables for normal distribution.
• Its characteristic function is

10
Exponential family

• Exponential family of distributions has the form
• Many distributions are special case of this
  family.
• Natural exponential family of distributions is
  the subclass of this family
• Where A(?) is natural parameter.
• If we use the fact that distribution should be
  normalised then characteristic function of the
  natural exponential family with natural parameter
  A(?) ? can be derived to be
• Try to derive it. Hint use normalisation fact.
  Find D(?) and then use expression of
  characteristic function and D(?) .
• This distribution is used for fitting generlised
  linear models.

11
Continuous distributions ?2

• Normal variables are called standardized if their
  mean is 0 and variance is 1.
• Sum of n standardized normal random variables is
  ?2 with n degrees of freedom.
• Density function is
• If there are p linear restraints on the random
  variables then degree of freedom becomes n-p.
• Characteristic function for this distribution is
• ?2 is used widely in statistics for such tests as
  goodness of fit of model to experiment.

12
Continuous distributions t and F-distributions

• Two more distribution is closely related with
  normal distribution. We will give them when we
  will discuss sample and sampling distributions.
  One of them is Students t-distribution. It is
  used to test if mean value of the sample is
  significantly different from 0. Another and
  similar application is for tests of differences
  of means of two different samples are different.
• Fishers F-distribution is distribution ratio of
  the variances of two different samples. It is
  used to test if their variances are different. On
  of the important application is in ANOVA.

13
Reference

• Johnson, N.L. Kotz, S. (1969, 1970, 1972)
  Distributions in Statistics, I Discrete
  distributions II, III Continuous univariate
  distributions, IV Continuous multivariate
  distributions. Houghton Mufflin, New York.
• Mardia, K.V. Jupp, P.E. (2000) Directional
  Statistics, John Wiley Sons.
• Jaynes, E (2003) The Probability theory Logic of
  Science