What to do when you are the only one in step

1
What to do when you are the only one in step

• Dr. M. Smith,
• S. M. I. L. E. Hardware / Software Co-design
  Laboratory,
• Dept. of Electrical and Computer Engineering,
• Dept. of Radiology, University of Calgary

2
Talk Overview

• Reason for doing the research
• Brief discussion of what everybody else was
  doing.
• Description of the little project we planned to
  do
• Our simulation study and all the problems that
  arose.
• Why so many problems?
• What we are currently doing (to solve the issue).
  

3
Story 1What you tell everybody else.

• Start of World War II with many men conscripted
  and being readied to be sent over-seas.
• After basic training, the men parade through the
  town (in front of their kin-folk) prior to
  embarking on a train.
• Mother (wife) and son watch the parade.
• Son wanting to believe in the perfection of his
  father
• Look, Mother! Father is the only one in step.

4
Story 2 -- Have confidence in yourself and your
research cability

• Sign on my desk
• given to me by one of my graduate students
• Its difficult being perfect
• Buts somebodys got to do it!

5
Background
HEMORRHAGIC
ISCHEMIC

• Stroke --the third leading cause of death and the
  leading cause of adult disability.
• Goal of therapeutic strategies is to minimize the
  progression of tissue damage in the acute phase
  of the disease.
• Methods to rapidly assess acute stroke in
  individual patients are highly desirable.

• 85 of the stroke cases are ischemic strokes due
  to a reduction of the blood supply by the
  presence of a clot in a feeding artery (adapted
  from www.lanacion.com).

6
Methods to measure Cerebral Blood Flow were known
(1996)

• Track a bolus of magnetic material through the
  brain (arterial and tissue signals)
• Convert changes in MR signal intensity to
  concentration curves using the magic log.
  Formula
• The technology of any sufficiently advanced
  civilization looks like magic. Arthur C. Clarke

7
What everybody else was doing

• Need to deconvolve tissue signal ( cVOI(t) ) by
  arterial signal ( cAIF(t) ) to get residue
  function ( R(t) ).
• Peak of residue function provides estimate of
  blood flow (CBF)

8
Clinical results Appear to make perfect sense
(Calamente, MRM, 2000)

• Impact of delay
• Impact of Dispersion

• a CBF map.
• b Signal intensity time
• A clear delay of 2 sec in the arrival of the
  bolus can be seen in the right side.
• The presence of such delay (and possibly
  dispersion) introduced a significant
  underestimation in the CBF map.
• The measured right to left ratio in the CBF map
  is 0.55 due to delay

9
What we were planning to tackle.Signal loss
through noise filtering
IMPACT OF NOISE FILTERING LOSS OF SIGNAL
10
Where did the signal loss come from?

• Deconvolution causes an enhancement of high
  frequency noise components.
• To stabilize the algorithm, you must apply a
  filter to reduce the noise.
• However, the noise filter also reduces the high
  frequency signal components so maximum of
  residue function is reduced CBF appears smaller

TIMEAMPLITUDELOSS
HIGHFREQUENCYLOSS
11
Plan of actionQuick one term project

• Step 1 - Stand on the shoulders of giants
  Repeat what everybody else is doing so we can
  check we understand the problem.
• Generate some artificial data (tissue and AIF)
• Add some noise
• Do deconvolution (standard approach) to get
  residue function.
• Noise filtering removes high frequency
  components
• Measure CBF as a function of delay / dispersion
  and tissue type

12
New idea based on a previously successful MRI
reconstruction approach

• Generate some artificial data (tissue and AIF)
• Add some noise
• Do deconvolution (standard approach) to get
  residue function.
• Noise filtering removes high frequency
  components
• MODEL the low frequency signal components and
  extrapolate those signals into high frequencies
  
• Compare our CBF to their CBF

13
ARMA modeling TERA algorithm

• Use known low frequency data to generate high
  frequency data

14
Issue 1 Insufficient information about how to
construct signals

• Mathematical formula for constructing arterial
  signal is given
• Nothing about how to construct tissue signal
  we suspect that either we are missing
  something obvious (out-of step) or else
  construction done by numerical convolution
  rather than algebraic.
• Nothing specific about how to add noise to get
  realistic data, although some people mention
  adding gaussian white noise to the
  concentration
• Every body discusses low and high signal to
  noise ratio but nobody says how to measure it.

15
Start putting on the engineers hat

• Generating data by convolution is a delicate
  process.
• If the data is not sampled fast enough then
  Nyquist is not satisfied.
• MR DSC data sampled at 2.25 seconds
• If Nyquist not satisfied then data gets
  distorted at high frequencies (aliasing).
• All CBF results are wrong, but by how much
  and when?

16
Other engineering stuffYou can get better
results doing it wrong

• Would everybody else not doing things the
  proper engineer way impact on our new method
  done the correct way?

17
2 -- We dont understand the properties of SVD
(time domain deconvolution)

• Need to deconvolve tissue signal ( cVOI(t) ) by
  arterial signal ( cAIF(t) ) to get residue
  function.
• Peak of arterial signal provides estimate of
  blood flow (CBF)

18
Use engineering principles again

• We would expect that frequency domain
  deconvolution to give same results as time domain
  deconvolution except for fine detail
• HOWEVER literature is saying MUCH BETTER
  RESULTS are being obtained with SVD than with
  FT does not make engineering sense unless
  something wonderful is happening

19
Problem 2 -- Noise modeling is being done wrong

• The MR signal (upper picture) has gaussian
  noise on it (unless very small in intensity and
  then the noise characteristics change)
• This means that adding noise to the concentration
  curves does not model clinical data

Added noise
Calculated noise
20
Paper 1 Discussing SNR issues based on true
noise model

• True SNR of concentration signal changes with MR
  signal intensity specific best conditions

21
Paper 1 Discussing SNR issues based on true
noise model

• Consequences we believe that everybody is
  setting the image parameters the wrong way

22
Paper 1

• Did not cause much controversy
• Other researchers have now demonstrated that our
  predictions are to be found in practice.
• Optimize SNR through TE changes and have
  different MR sequence for tissue and AIF signals
• Largely ignored
• Difficult to get the correct imaging
  parameters.
• Takes too long to get an DSC image sequence
• Tissue signal have low intensity, therefore
  people push arterial signals into an
  unsatisfactory high intensity region to
  compensate.

23
Next step -- Deconvolution

• We have the noise simulation problems understood
• Lets try using frequency domain deconvolution
  (about which we have much knowledge) rather than
  SVD time domain deconvolution
• As engineers we expect Equivalent results
  between SVD and FT

24
Trouble is the FT and SVD answers are very
different

• FT shows no time delay effects that are so
  evident with SVD. We are really out of step

SVD deconvolution
FT deconvolution
25
Noise characteristics of SVD and FT differ in
unexpected way

• Noise Enhancement during deconvolution

SVD deconvolution eigen-value thresholding causes
band pass filtering
26
We have big problems

• The delay sensitivity of SVD deconvolution is
  breaking the deconvolution rules
• BUT the SVD is a VERY well-known algorithm and
  NOBODY has reported problems like this in 50
  years
• The noise effect shows that the SVD filtering is
  a series of band pass filters.
• Band pass characteristics controlled by
  eigenvalues which are identical to the
  (ordered) Fourier transform coefficients of the
  arterial function
• This was found empirically by us, but turns out
  to be well-known effect from radar studies in
  1991

27
Engineering convolution theory indicates we
are right

• Consider convolving (or deconvolving) two signals
• LINEARITY PROPERTY
• Double the amplitude of one input doubles
  output amplitude no change in shape
• POSITION INDEPENDENT
• Shift position of input by amount x. Output will
  shift position by amount x no change in shape
• Theory indicates that a proper deconvolution
  algorithm should be delay independent

28
SVD well known Why is it not working in DSC MR
studies?

• Actually neither SVD nor FT have ever really
  worked in one sense but nobody says it.
• Deconvolution works by deconvolving the effect
  by its cause and a cause signal always
  arrive before the effect.
• The tissue is not the effect that is produced
  by the arterial signal, but is the effect of
  the injection into the arm.
• Thus it is physiologically possible for the
  tissue effect signal to arrive BEFORE the
  proxy arterial cause signal.

29
SVD and FT deconvolution have different properties
NEGATIVE POSITIVE TIME TIME

• The FT deconvolution algorithm has cyclic
  properties
• In the presence of a delay, any negative time
  residue function signals are wrapped around
  (aliased) to become a false high time signal.
• However, PROVIDED THERE ARE NO TRUE HIGH TIME
  SIGNALS, we can unwrap and get correct answer .

UNWRAPPED HIGH TIME SIGNAL
30
SVD and FT deconvolution have different properties
NO NEGATIVESIGNAL ALLOWED

• The SVD deconvolution algorithm was not being
  implemented with cyclic properties
• No negative time signals are allowed.
• But that energy must go somewhere and it goes
  into boosting the early residue function peak
• For a zero delay -- This boost counterbalances
  the signal loss from noise filtering
• SVD acts as the better algorithm when
  incorrectly implemented
• However, the improvement is very unstable

MISPLACED NEGATIVE ENERGY
31
Big fight with reviewers

• First of all reviewers would not accept that
• There was an effect or
• that our theory was valid
• Later, when somebody well known published a
  circular SVD implementation, we were told by the
  reviewers that since a better algorithm had
  already been published, then ours should not be
  published.
• Fortunately the editor stepped in and we
  published our improved SVD algorithm (as a short
  note), but we never recovered the precedence.
• New papers are still showing misunderstanding of
  the significance of what we have explained about
  delay issues.

32
0ther implications

• All that dispersion effect is also an artifact

Using a delay insensitivedeconvolution
approach shows dispersion effect is much
smaller than described earlier
33
Biggest issue remaining

• We are continually changing our algorithms as we
  better understand the engineering theory.
• How can we (easily) check that the changes we are
  making are not having an unexpected effect in
  previously working parts of our code.
• In the business world, a new concept in software
  development is Agile a light weight,
  low-document producing development process.
• A key element of Agile is test driven
  development and an automated testing framework
  two issues useful in different ways

34
Comparing Test-Driven-Development with the
Scientific method (Mugridge 2003)
The scientific method
Test-Driven Development (TDD)
We dont need to change our thought processes
very much to switch to TDD. Biggest issue is
having to change our work habits and beliefs. As
a physicist I had been trained to think about
tests and testing issues before coding,
therefore formalizing those thoughts into real
tests is not too hard (30 of the time)
35
Main difference between TDD and normal software
development
TDD approach -- Many initial testsused to
describe ideas later used for regression
testing when ideas change
Standard water-fallmethod. Tests often forgotten
in time crunch.
36
We have been successfull in applying TDD to
biomedical embedded systemsF. HuangA. TranA.
Kwan
37
Current research (J. Qiao)

• How do you move the idea behind applying the
  scientific method in planning your research
  procedure over intousing
  test-driven development in planning the software
  code (Matlab) you need for that research
  procedure and later use those tests when
  commercializing onto the biomedical instrument?

38
Conclusion

• When starting your research project make sure
  you understand your goals.
• Be prepared to change your goals as opportunities
  arise.
• Try to duplicate the results in existing
  literature, but remember, you are engineers and
  have a different knowledge set that many of the
  clinical people
• Be prepared for unexpected results.
• Have an automated testing approach so that you
  can duplicate your (software) results easily and
  provide easily repeatable evidence that
  everybody else has not handled things correctly.