IRAC Pipeline Data Analysis and Pipeline Validation Plan

1
IRAC Pipeline Data Analysis and Pipeline
Validation Plan

• Jason Surace
• January 26, 2001

2
Pipeline Module Testing
Nearly all IRAC pipeline modules have been tested
in some way with real data, i.e. produced by
the flight instrument.

• But, testing with real data has been somewhat
  haphazard and not very systematic.
• We have had to sift through 10s of Gb of test
  data.
• Instrument not completed or in stable environment
  - not clear what is best characterization data.
• Analysis has been lengthy - lots of problems have
  been turned up both by us and SAO-IT. IT is still
  analyzing data and working on characterizing
  instrumental signatures.
• Short on manpower. IT pipeline work pre-empted by
  instrument delivery.

How to reach a point where we can check off each
module as finished?
3
Pipeline Validation Plan
1) Instrument team selects and provides a
validation data set, where they have
pre-analyzed the data. 2) SSC processes this raw
data via pipeline. 3) This data is then analyzed
by SSC-IST/SAO-IT. 4) If results match, were
done. If not, we go back to IT and iterate by
asking for a new algorithm or a clarification of
requirements.
In this way, the results of each module and
thread will be validated and approved by IT. The
pipeline has renewed visibility currently with IT
as instrument construction ramps down.
4
Example - LINCAL
IRAC data must be corrected for non-linear
response of detector. Science requirement is a
very stringent 1 linearity over usable data
range of detector.
Previous LINCAL effort based on a multivariate
function delivered by SAO S'S (C A/Sqrt(B-S))
This was tested using the comprehensive
performance test (CPT) data generated via ASIST
at GSFC. This was intended to be the definitive
test data set.
5
Early Analysis Problems
However, testing with CPT data showed data was
too unstable to demonstrate requirement. Camera
was turned on and off each day during multiday
test, may have been other problems.
Normalized mean count levels for supposedly
identical linearity frames taken on different
days during CPT. Note that on a given day the
stability is very high (drift lt 0.5 per 6 hours).
6
Validation Data
At SSC request, Rochester provided a new
linearity dataset taken under laboratory
conditions which they believe can demonstrate the
1 requirement. Jason writes new software to
analyze this dataset and prototype several
different ways to linearize the data.
Derived linear part (flat-fieldlamp pattern)
Derived non-linearity for rate method.
Input TCAL with increasing exptimes
(See http//humu.ipac.caltech.edu/sirtf/irac/fligh
t/lincal/lincal_analysis.html)
7
A Comparison of Methods
error as a function of full-well depth for
individual pixels. LINCAL (blue) shows behavior
seen in previous tests. Scatter and offset are
never below a few percent and hence fails the
requirement - functional form does not fit well .
Red is a new proposed quadratic solution.
Both models break down completely within 10 of
full-well capacity.
8
So We Redo It
New solution will be a quadratic function S' A
(-Asqrt((A2)-(4BS)))/(2 B))
Quadratic solution expected to exceed 1
requirement up to 90 full-well capacity.
9
The Next Iteration
There is a plan to acquire new linearity test
data during CTA testing. We have a more uniform
illuminator available, and the data will be taken
in a flight-like manner and environment. The new
LINCAL will be ready to test this once the data
arrives. Rec/Del for all IRAC algorithms and
data from SAO-IT is 02/01/01.