Statistical Processes for Time and Frequency A Tutorial Victor S. Reinhardt 101701

1
Statistical Processes for Time and FrequencyA
TutorialVictor S. Reinhardt10/17/01
2
Statistical Processes for Time and
Frequency--Agenda

• Review of random variables
• Random processes
• Linear systems
• Random walk and flicker noise
• Oscillator noise

3

• Review of Random Variables

4
Continuous Random Variable
Ensemble of N Identical Experiments
Unpredictable Result
xN

• Random Variable x
• Repeat N identical experiments Ensemble of
  experiments
• Unpredictable (Variable) Result xn
• Nx Number of of times value xn between x and
  xdx
• Probability density function (PDF) or
  distribution p(x)

x3
x2
x1
5
PDF and Expectation Values

• Range of random variable x from a to b
• Mean value ?x
• Standard variance ?d2x
• Standard deviation ?dx

6
Probability Distributions

• Gaussian (Normal) PDF
• Range (-?, ?) Mean m
• Standard deviation sd
• Uniform
• Range (-D/2, D/2) Mean 0
• Standard deviation D/120.5
• Examples Quantization error, totally random
  phase error

Pgauss(x)
x
Puniform(x)
x
-D/2
D/2
0
7
Statistics

• A statistic is an estimate of a parameter like m
  or s
• Repeat experiment N times to get x1, x2, xN
• Statistic for mean ?x is arithmetic mean
• Statistics for standard variance ?dx
• Standard Variance(m known a priori)
• Standard Variance(with estimate of m)
• Good Statistics
• Converge to the parameter as N ? ? with zero
  error
• Expectation value parameter value for any N
  (Unbaised)

8
Multiple Random Variables

• x1 and x2 two random variables (1 and 2 not
  ensemble indices but indicate different random
  variables)
• Joint PDF p(2)(x1,x2) (2) means 2-variable
  probability
• Expectation value
• Single Variable PDF
• Conditional PDF p(x1x2) is PDF of x2 occuring
  given that x1 occurred
• Mean Covariance matrix
• Statistical Independence
• p(2)(x1,x2) p(1)(x1)p(1)(x2)
• Then

(k k 1,2)
9
Ensembles Revisited

• The ensemble for x is a set of statistically
  independent random variables x1, x2, .. xN with
  all PDFs the same p(1)(x)
• Thus

Ensemble of N Identical Experiments
Each with same PDF p(x)
xN
Eachstatistically independent
x3
x2
x1
(F2 for normal distribution)
10

• Random Processes

11
Random Processes

• A random function in time u(t)
• Is a random ensemble of functions
• That is defined by a hierarchy probability
  density functions (PDF)
• p(1)(u,t) 1st order PDF
• p(2)(u1,t1 u2,t2) 2nd order joint PDF
• etc
• One can ensemble average at fixed times
• Or time average nth member

12
Time Averages and Stationarity

• Time mean
• Autocorrelation function
• Wide sense stationarity
• Strict Stationarity
• All PDFs invariant under tn ? tn - t
• Ergodic process
• Time and ensemble averages equivalent

( 0 for random processes we will consider)
13
Types of Random Processes

• Strict Stationarity All PDFs invariant under
  time translation (no absolute time reference)
• Invariant under tn ? tn - t (all n and any t)
• Implies p(1)(x,t) p (1)(x) independent of
  time p(2)(x1,t1 x2,t2) p(2)(x1,0 x2,t2- t1)
  function of t2- t1
• Purely random process Statistical independence
• p(n)(x1,t1 x2,t2 .xn,tn) p(1)(x1,t1) p
  (1)(x2,t2) .. p (1)(xn,tn)
• Markoff Process Highest structure is 2nd order
  PDF
• p(x1,t1...xn-1,tn-1 xn,tn) p(xn-1,tn-1
  xn,tn)
• p(x1,t1 ...xn-1,tn-1 xn,tn) is conditional PDF
  for xn(tn) given thatx1(t1) ...xn-1,tn-1 have
  occurred
• p(n)(x1,t1 x2,t2 .xn,tn) p(1)(x1,t1)
  p(xk-1,tk-1 xk,tk)

14

• Linear Systems

15
Linear Systems

• In time domain given by convolution with response
  function h(t)
• Fourier transform to frequency domain
• The fourier transform of the output is

Time Domain
Frequency Domain
16
Single-Pole Low Pass Filter
C
t1 R1C
t2 R2C
R2
U(f)
-
V(f)
G-1
R1

(Causal filter)
(3-dB bandwidth)
17
Spectral Density of a Random Process

• Requires wide sense stationary process
• The spectral density is the fourier transform of
  the autocorrelation function
• For linearly related variables given by
• The spectral densities have a simple relationship

V(f) jw U(f)
Sv(f) w2Su(f)
18
Average Power and Variance

• Autocorrelation Function back from Spectral
  Density
• Average power (intensity)
• Average power in terms of input
• For ergodic processes
• Where sd2 the standard variance is

(Mean is assumed zero)
19
White Noise

• Uncorrelated (zero mean) process
• Generates white spectrum
• At output
• Bn is noise bandwidth of system

• For Single-Pole LP Filter
• Bn ? B3-dB as number of poles increases
• For Thermal (Nyquist) Noise
• No kT

20
Band-Limited White Noise Correlation Time

• White noise filtered by single pole filter
• t1 t2 to
• Called Gauss-Markoff Process for gaussian noise
• Frequency Domain
• Time Domain
• Correlation Time to
• Correlation width Dt 2to

21
Spectrum Analyzers and Spectral Density

• Model of Spectrum Analyzer
• Downconverts signal to baseband
• Resolution Filter BW Br
• Detector
• Video Filter averages for T 1/(2Bv)
• Spectrum Analyzer Measures Periodogram (Br?0)
• uT(t) Truncated data from t to tT
• Fourier Trans
• Wiener-Khinchine Theorem
• When T ? ?
• Periodogram ? Spectral Density

Radiometer Formula (finite Br)
Same as
22
Response Function and Standard Variance for Time
Averaged Signals

• Finite time average over t
• Response Fn for average
• Variance of with H1
• For
• So s12 diverges when

y (f-fo)/fo
v(t) t,t
Response Function
Sy(f)??
H1(f)2
0
( for non-stationary noise)
23
Response Function for Zero Dead-Time Sample
(Allan) Variance

• Response for difference of time averaged signals
• Variance with H2 (Allan variance)
• For
• So s22 doesnt diverge for

y (f-fo)/fo
v(t) tT,t - t,t
Response Function
1/t
h2(t)
t
tt
t2t
-1/t
( for noise up to random run)
24
Graphing to Understand System Errors

• Can represent system error as
• h(t) includes
• Response for measurement
• Plus rest of system
• Graphing h(t) or H(f) helps understanding
• Example Frequency error for satellite ranging
• Ranging sd2(t,T) s22(T,t) Allan variance
  with dead time t and averaging time T reversed
• Radar sd2(t,T) s22(t,T) no resversal of T
  and t

(t T)
h(t)
tTt
T
1/t
t
t
-1/t
t
tT
tt
v(t) tT,t - t,t
25

• Random Walk and Flicker Noise

26
Integrated White Noise--Random Walk (Wiener
Process)

• Let u(t) be white noise
• And
• Then
• where t
• Note Rv is not stationary (not function of t-t)
• This is a classic random walk with a start at t0
• The standard deviation is a function of t

Random Walk Increases as t½
27
Generating Colored Noise from White Noise

• A filter described by h(t-t) is called a Wiener
  filter
• Must know properties of filter for all past times
• To generate (stationary) colored noise can Wiener
  filter white noise
• Can turn convolution into differential
  (difference) equation (Kalman filter) for
  simulations

White Noise
Wiener Filter
Colored Noise
28
Wiener Filter for Random Walk
C
t1 R1C
e R1/R2
U(f)
R2
-
V(f)
G-1
R1

29
Wiener Filter for Flicker Noise

• Impedance of diffusive line
• White current noise generates flicker voltage
  noise
• Ni Current noise density

Heavyside Model of Diffusive Line
R
R
Z
C
C
Impedance Analysis
R
Z
Z
C
Flicker Voltage Noise
White Current Noise
v(t)
i(t)
30
Multiplicative Flicker of Phase Noise

• Nonlinearities in RF amplifier produce AM/PM
• Low frequency amplitude flicker processes
  modulates phase around carrier through AM/PM
• Modulation noise or multiplicative noise is what
  appears around every carrier

AM/PM converts low frequency amplitude
fluctuations into phase fluctuations about
carrier
31
An Alternative Wiener Filter for Flicker Noise

• Single-Pole Filters T. C. t
• Independent current sources
• Integrate outputs over t

N Independent White Current Noise Sources
C
t RC ?-1
White Current Source I(f)
R Constant
R
V(f)
-
G-1

SI(f)I2
Filter t1?0
Filter t2
Filter tN??
Sum (Integrate) Over Outputs
Flicker Noise
32
A Practical Wiener Filter for Flicker Noise

• Single-pole every decade
• With independent white noise inputs
• Spectrum
• For time domain simulation turn convolutions into
  difference equations for each filter and sum

Error in dB from 1/f
33

• Oscillator Noise

34
Properties of a Resonator

• High frequency approximation (single pole)
• ?f 3-dB full width
• Phase shift near fo

35
Simple Model of an Oscillator

• Amplifier and resonator in positive feedback loop
• Amplifier
• Amp phase noise
• Sf-amp (f) FkT/Pin (1 ff/f)
• Thermal noise flicker noise
• Resonator (Near Resonance)
• fR -2QLy y (f - fo)/fo
• Oscillation Conditions
• Loop Gain GaGL ? 1
• Phase shift around loop 0
• fR famp 0

Resonator
Near Resonance fR -2QLy
Loss GR YR
Loaded Q QL
Flicker of Phase
Thermal
Oscillation Conditions GaGR Loop Gain ?
1 S f Around Loop 0
Pin
Amp
Noise
Gain Ga Phase Shift famp Noise Figure
F Flicker Knee ff
White Noise Density FkT
36
Leesons Equation

• Phase Shift Around Loop 0
• famp 2QLy - fR
• Thus the oscillator fractional frequency y must
  change in response to amplifier phase
  disturbances famp
• Amp Phase Noise is Converted to Oscillator
  Frequency Noise
• Sy-osc(f) 1/(2QL)2 Sf-amp(f)
• But y wo-1df/t so
• Sf-osc(f) (fo2/f2) Sy-osc(f)
• And thus we obtain Leesons Equation
• Sf-osc(f) ((fo/(2QLf))21)(FkT/Pin)(1 ff/f)

Resonator Phase vs df/f Response
The Oscillator df/f must shift to compensate
for the amp phase disturbances
37
Oscillator Noise Spectrum

• Oscillator noise Spectrum
• Sf(f) K3/f3 K2/f2 K1/f K0
• Some components may mask others
• Converted noise
• K2 FkT/Pin (fo/(2QLf))2
• K3 FkT/Pin(ff/f) (fo/(2QLf))2
• Varies with (fo/(2QL)2 and FkT/Pin
• Original amp noise
• Ko FkT/Pin
• K1 FkT/Pin(ff/f)
• Only function of FkT/Pin
• and flicker knee

Oscillator Noise Spectrum
Leesons Equation
Sf-osc(f) (fo/(2QLf))21)(FkT/Pin)(1 ff/f)
38
References

• R. G. Brown, Introduction to Random Signal
  Analysis and Kalman Filtering, Wiley, 1983.
• D. Middleton, An Introduction to Statistical
  Communication Theory, McGraw-Hill, 1960.
• W. B, Davenport, Jr. and W. L. Root, An
  Introduction to the Theory of Random Signals and
  Noise, Mc-Graw-Hill, 1958.
• A. Van der Ziel, Noise Sources, Characterization,
  Measurement, Prentice-Hall, 1970.

• D. B. Sullivan, D. W. Allan, D. A. Howe, F. L.
  Walls, Eds, Characterization of Clocks and
  Oscillators, NIST Technical Note 1337, U. S.
  Govt. Printing office, 1990 (CODENNTNOEF).
• B. E. Blair, Ed, Time and Frequency Fundamentals,
  NBS Monograph 140, U. S. Govt. Printing office,
  1974 (CODENNBSMA6).
• D. B. Leeson, A Simple Model of Feedback
  Oscillator Noise Spectrum, Proc, IEEE, v54,
  Feb., 1966, p329-335.