How Extracting Information from Data Highpass Filters its Additive Noise

1
How Extracting Information from Data Highpass
Filters its Additive Noise

• Victor S. ReinhardtRaytheon Space and Airborne
  SystemsEl Segundo, CA, USA
• PTTI 2007

Thirty-Ninth Annual Precise Time and Time
Interval (PTTI) Systems and Applications Meeting
November 26 - 29, 2007Long Beach, California
2
Various Measures of Random Error Are Used Across
EE Community

• Mean Square (Observable)Residual Error
• After removing a modelfunction estimate of
  truecausal behavior of data
• Jitter (and Wander)
• Residual errors HP LP filtering
• Difference (?) Variances
• Allan ? Mean sq of ?(?)y(t)
• Hadamard or Picinbono? Mean sq of ?(?)2y(t)

CausalEstimate
v(t)?x(t),y(t),?(t)
3
Some Common Wisdoms About These Error Measures

• Residual error diverges in presence of negative
  power law (neg-p) noise
• Power law noise ? PSD Lv(f) ? f p
• Neg-p ? p lt 0 (-1,-2,-3,-4) ?White noise ? p 0
• Jitter ?-variance used to correct this problem
• ?-variance doesnt measure same thing as MS
  residual error
• Need real variance not Allan Variance
• Will show common wisdoms are not true
• And all 3 essentially measure same thing for
  polynomial models of causal behavior

?-Variances ? Residual Error ? Jitter
4
Will Demonstrate This by Showing

• The estimation of causal behavior from data HP
  filters noise in residual error
• Res error in general converges for neg-p noise
• Res error guaranteed to converge if free to
  choose model for causal behavior
• Can define jitter simply as observable residual
  error
• ?-variances are measures of residual error
• For any of samples
• When causal model is a polynomial

?-Variances ?
Residual Error ? Jitter
5
Residual Error

• Consider N data samples of data over observation
  interval T
• Data v(tn) will be modeled as
• True causal function vc(tn) plus
• True noise vp(tn) with Lv(f) ? f p
• vp(tn) also true residual error
• But not measurable from data over T

v(tn)
True Causal Function vc(t)
Also True Residual Error
6
Residual Error

• Can estimate residual error by fitting model
  function vw,M(t,A) to data
• A (ao,a1,aM-1) ? M adjustable parameters
• A ? Information extracted from data
• Class of vw,M(t,A) ? (M-1)th order polynomials
• Will use Least SQ Fit to estimate vc(tn)
• LSQF equivalent to many other methods

v(tn)
vc(t)
7
Residual Error

• Observable residual error ? data est fn
• vj(tn) v(tn) - vw,M(tn,A)
• Jitter ? vj(tn) with no ad hoc filtering
• True function error ? est true functions
• vw(tn) vw,M(tn,A) vc(tn)
• vw(tn) ? wander (Not observable from data over T
  alone)

v(tn)
Model Fn Est vw,M(t,A)
vc(t)
? ?v-j2 MS of vj(tn)
? ?v-w2 MS of vw(tn)
8
Why Residual Error is Important Measure

• Directly relates lower level error to primary
  performance measures in many systems
• SNR ?BER ?MNR ?NPR ?ENOB
• Must use observable residual error for
  verification with data over T
• For true errors (over T) must measure over gtgt T
  for neg-p noise (exception to be discussed)

v(tn)
Model Fn Est vw,M(t,A)
vc(t)
9
HP Filtering of Noise Due toInformation
Extraction

• Proved in paper
• Will explain graphicallyas follows
• For white noise LSQF behaves in classical manner
• ?v-w ? 0 as N ? ?
• Note T fixed as N varies
• But for neg-p noise
• ?v-w not ? 0 as N ? ?
• Because fitted vw,M(t,A) tracks highly correlated
  LF noise components in data

long term error-1.xls
10
HP Filtering of Noise Due toInformation
Extraction

• Happens because LSQF cant separate highly
  correlated LF noise
• With Fourier freqs f 1/T
• From the causal behavior
• True for all noise
• Implicit in LSQF theory
• Only apparent for neg-p noise because most power
  in f 1/T
• This tracking causes HP filtering of noise in vj
  ?v-j

long term error-1.xls
11
HP Filtering of Noise in Spectral Integral
Representation of ?v-j2

• Hs(f) System response function
• Described in Reinhardt, FCS 2006
• Hs(f)2 is used to replace upper cut-off freq fh
• Can show Hs(f) often HP filters Lv(f)(as well as
  LP filters) ? Helps ?v-j2 converge

12
HP Filtering of Noise in Spectral Integral
Representation of ?v-j2

• Kv-j(f) spectral kernel also HP filters noise
• Paper proves Kv-j(f) ? f 2M (fltlt1)
• When va,M(t,A) is (M-1)th order polynomial
• Kv-j(f) at least ? f 2 (fltlt1)
• For any va,M(t,A) with DC component
• Thus convergence of ?v-j2 depends on complexity
  of model function va,M(t,A)
• Convergence Guaranteed if free to choose model
  function

13
HP Filtering of Noise in Spectral Integral
Representation of ?v-j2

• ?v-c2 Contribution due to model error
• Model error occurs when model function cannot
  follow variations in vc(t) over T
• If model error present ?v-j2 ?v-w2 will
  increase
• Thus if model function cant track causal
  behavior over T
• The causal behavior will contaminate?v-j2 (
  ?v-w2)

14
Simulation Verifying Kv-j(f) ? f 2M (fltlt1) for
Polynomial va,M(t,A)
? f 2
HP KneefT ? 1/T
? f 4
Kv-j(f) in dB
? f 8
? f 10
? f 6
SimulationResults ?M lt 1E-3
Log10(fT)
(N1000)
(Unweighted LSQF)
15
Jitter and Wander

• Defined by ITU, IEEE BTS, SMPTE
• Filtered residual x-error after removing causal
  (time) freq offset freq drift
• Jitter ? HP filtered (gt fc)
• Wander ? LP filtered (lt fc)
• Problem relating fc to user system params
• ITU ? fc 10 Hz
• Standardizes HW producers
• But not related to parameters in user systems
• IEEE BTS, SMPTE ? fc PLL BW in system
• What about systems without PLLs?

16
HP Filtering of Residual Error Resolves fc
Relationship Problem

• Jitter ? Observable residual error
• vj(tn) v(tn) - vw,M(tn,A)
• Wander ? True causal function error
• vw(tn) vw,M(tn,A) vc(tn)
• Have property ? vj(tn) vw(tn) vp(tn)
• These definitions apply to any type of causal
  function removal any variable
• HP LP properties generated by system

17
HP Filtering of Residual Error Resolves fc
Relationship Problem
f

• LSQF HP filters vj LP filters vw ? fT ? 1/T
• Hs(f) filters both the same ? fl HP fh LP
• For as T?? (fT ltlt fl) wander shrinks to zero
• If Hs(f) alone can overcome pole in Lv(f)
• So wander can also converge for neg-p noise
• Jitter variance with brickwall fc fhis just
  bandpass approximation of ?v-j2

18
?-Variances as Measure of ?v-j2 When va,M(t,A) is
Polynomial

• ?v,M(?)2 ?M x Mean Sq of ?(?)Mv(tn)
• ?(?)v(t) v(t?) - v(t)
• ?(?)2v(t) v(t2?) - 2v(t?) v(t)
• ?M ? All ?v,M(?)2 are equal for white noise

?v,M(?)2 ?MMS?(?)Mv(tn)
? MS over N samples in T ?
19
?-Variances as Measure of ?v-j2 When va,M(t,A) is
Polynomial

• Paper shows
• For ?v-j2 ? unbiased mean square
• Unbiased ? Divide sum of squares by N - M
• Well-known for Allan (2-sample) variance
• Is 2-sample MS residual of y(t) with 0th order
  polynomial (freq offset) removed
• Hadamard-Picinbono variance is 3-sample residual
  variance of y(t) for M 2
• 1st order polynomial (freq offs drift) removed

?v-j2(NM1) ?v,M(?)2 when ? T/M
20
Can Extend Equivalence to Any N

• Can show biased RMSvj doesnt vary much with N
• Biased ? Divide sum sq by N
• Note T fixed as N varied
• Thus for any N can writeunbiased ?v-j2 as
• Approx true for any p
• Can generate exact relationship for each p like
  Allan-Barnes bias functions

21
Consequences of ?v,M(?) as Approximate Measure of
?v-j

• Justifies using ?v,M(T/M) in residual error
  problems
• When vw,M(t,A) is (M-1)th order polynomial
• Dont have to perform LSQF on data if use ?v,M(?)
  to estimate ?v-j
• Because ?(?)M of (M-1)th order polynomial 0
• Well-known insensitivity of ?v,M(?) to (M-1)th
  polynomial causal behavior
• True even if there is model error (effect on both
  ?v,M(?) ?v-j the same)

22
Consequences of ?v,M(?) as Approximate Measure of
?v-j

• Provides guidance in determining order of ?v,M(?)
  to use as stability measure
• vw,M(t,A) Polynomial aging function
• Aging is strictly fitted over T M?
• For ? decoupled from T/M?v,M(?) ? ?v-j approx
  true
• Causal terms not modeled in va,M(t,A) are part of
  instability measured by ?v,M(?)
• Explains sensitivity of Allan variance to freq
  drift (only freq offset modeled)
• insensitivity of Hadamard variance to such
  drift (both freq offset drift modeled)

23
x y Difference Variances Interpreted as Aging
Removed ?v-j2
M ? Var of y Aging ExcludedApplication ? Var of x Aging ExclApplication
0 MSy NoneSynthesizers Rel time dist equip MSx NoneAbs time dist equip
1 Allan y y offsetOscillators (y drift in instability) TIErms2/2MSTIE/2 x offsetSynth rel time dist
2 Hadamard Picinbono y ofs driftOscillators (y drift not in instability) Allan xJitter 2 x y offsetOsc (y drift in instab)
3 ?x,32(?) is equivalent toMS of x-jitter with time freq offsets freq drift removed ?x,32(?) is equivalent toMS of x-jitter with time freq offsets freq drift removed Hadamard Picinbono x,y ofs y driftOsc (y drift not in instab)

• ssss

24
x y Difference Variances Interpreted as Aging
Removed ?v-j2
M ? Var of y Aging ExcludedApplication ? Var of x Aging ExclApplication
0 MSy NoneSynthesizers Rel time dist equip MSx NoneAbs time dist equip
1 Allan y y offsetOscillators (y drift in instability) TIErms2/2MSTIE/2 x offsetSynth rel time dist
2 Hadamard Picinbono y ofs driftOscillators (y drift not in instability) Allan xJitter 2 x y offsetOsc (y drift in instab)
3  ?x,32(?) is equivalent toMS of x-jitter with time freq offsets freq drift removed  ?x,32(?) is equivalent toMS of x-jitter with time freq offsets freq drift removed Hadamard Picinbono x,y ofs y driftOsc (y drift not in instab)

• Explains why low order ?v,M(?) used for
  synthesizers distribution equipment
• Uncontrolled (but fixed) x y offsets are part
  of random error
• Not considered part of random error for
  oscillators

25
x y Difference Variances Interpreted as Aging
Removed ?v-j2
M ? Var of y Aging ExcludedApplication ? Var of x Aging ExclApplication
0 MSy NoneSynthesizers Rel time dist equip MSx NoneAbs time dist equip
1 Allan y y offsetOscillators (y drift in instability) TIErms2/2MSTIE/2 x offsetSynth rel time dist
2 Hadamard Picinbono y ofs driftOscillators (y drift not in instability) Allan xJitter 2 x y offsetOsc (y drift in instab)
3 ?x,32(?) is equivalent toMS of x-jitter with time freq offsets freq drift removed ?x,32(?) is equivalent toMS of x-jitter with time freq offsets freq drift removed Hadamard Picinbono x,y ofs y driftOsc (y drift not in instab)

• Hadamard-Picinbono variance or ?x,32(?)
  equivalent to MS of x-jitter with time freq
  offsets freq drift removed

26
What to Do When ?v-j Does Diverges

• ?v-j or ?v,M(?) can diverge for neg-p noise
• When vw,M(t,A) is fixed by system spec or
  physical problem being addressed
• Has been considered mathematical nuisance to be
  heuristically patched
• Such a divergence indicates real problem
• In system design, specification, or analysis
• vw,M(t,A) fixed in spec for a reason
• Not free to change w/o changing problem
• Thus divergence is diagnostic indication of a
  system problem to be fixed

27
Divergence Example 1st Order PLL in Presence of
f -3 Noise

• Well-known that 1st order PLL will cycle slip
  when f -3 noise is present
• Indicated by divergence in ??-j for M0
• Changing to M gt 0 ??-j eliminates divergence
• But this will not stop the cycle slipping

28
Can Only be Fixed by Changing System Design or
Spec

• Fix design ? Eliminate slips
• By changing design to 2nd order PLL
• Fix spec ? Tolerate cycle slips but must change
  ??-j spec to exclude cycle slipped data
• Effectively changes Kv-j(f)
• Should also specify mean time to cycle slip
• Non-essential divergences
• System OK but wrong Hs(f) or Kv-j(f) due to
  faulty analysis
• Example Failure to recognize HP filtering of
  residual error

29
Final Summary Conclusions

• Jitter, residual error, and ? variances can be
  viewed as equivalent measures
• When polynomial used for causal model
• Residual error guaranteed to converge if free to
  choose model for causal function
• Because order of HP filtering increases with
  complexity of model function used to estimate
  causal behavior
• Residual error often converges even when model
  function fixed by spec or problem
• When doesnt is indication of real problem in
  design, spec, or analysis of system in question

30
Final Summary Conclusions

• Jitter should be defined as residual error
  without ad hoc HP filtering
• Because causal extraction provides HP filtering
• True causal error (wander) only accessible by
  determining true noise in system
• Paper is generalization consolidation of
  previous work by many authors
• Allan, Barnes, Gagnepain, Vernotte, Greenhall,
  Riley, Howe, many others
• Preprints www.ttcla.org/vsreinhardt/