Pan-STARRS%20Image%20Processing%20Pipeline

1
Pan-STARRS Image Processing Pipeline
An Overview
IFA Pan-STARRS Seminar 735
October 6, 2004
2
Summary of Topics

• Image Analysis Steps / Science Data Products
• IPP System Architecture Motivations
• Software Organization (PSLib / Modules /
  Analysis Stages)
• PS-1 vs PS-4 implications for IPP

3
IPP within Pan-STARRS
Telescopes
OTIS
Cameras
raw images
metadata
metadata, detections
metadata, detections
MOPS Science Client
IPP
IDs
IDs
Legend
orbits identifications
PS Subsystem
External System
metadata, detections
static sky images
PSPS
pixel data
photons
filtered detections metadata
meta object data
commands
other data
static sky images
World Users
Solar System Community
4
IPP Image Processing Responsibilities

• Processing of single images (phase 2)
• remove instrumental signature
• bright object detection / basic characterization
• initial astrometric photometric calibration
• Merging of image groups (phase 4)
• warp resample to reference image grid
• stack and CR-reject
• subtract static sky image
• measure objects on difference summed images
• update static sky
• Construct Calibrations
• basic calibration images (bias, dark, flat, etc)
• derived calibration images (fringe, flat-field
  corrections, etc)
• other calibration data (optical distortion
  parameters, zero-points, etc
• Analysis Trace Monitor
• System Characterization

5
Phase 2 Issues

• Flat-fields are corrected based on stellar
  photometry

6
Phase 2 Issues

• Flat-fields are corrected based on stellar
  photometry
• fringe frames may be built with a monochromatic
  dome source
• fringe correction may be based on atm. emission
  line observation

7
Phase 2 Issues

• Flat-fields are corrected based on stellar
  photometry
• fringe frames may be built with a monochromatic
  dome source
• fringe correction may be based on atm. emission
  line observation
• minimal object parameters (x,y, mag,
  stellar/non-stellar, basic shape)
• high detection threshold (20 sigma?)
• we keep postage-stamp images on a (small) subset
  of objects

8
Phase 4 Issues

• overlapping pixels are mapped to common (sky)
  reference frame
• Object detections are perfomed on
• Difference Images (P4D) to low threshold (3
  sigma)
• Improved Summed Images (P4S) to modest threshold
  (5-10 sigma)
• Static Sky is updated
• note caveat in CR rejection for sequential vs
  simultaneous images

Phase 2 images 0.3 arcsec/pix Chip/Cell units
Phase 4 images 0.2 arcsec/pix Static Sky cells

-

-
cleaned
static sky
difference image (P4D)
also yields transient-free summed image (P4S) and
updated static sky image.
9
Static Sky Analysis

• Detailed object analysis including
• complex object deblending a la SDSS
• simultaneous multi-band analysis
• Analysis is NOT performed nightly
• Static Sky changes slowly
• Goal is deep science, not variability
• Analysis is computationally more intense than
  P2,P4
• Rolling Analysis
• 1 degree RA strips per day
• only RA within 10 degrees of solar RA

10
IPP Data Products

• Imaging Data Products
• Static Sky images
• Postage Stamp Images
• Object Measurements
• P2, P4S, P4D, Static Sky
• rudimentary object associations
• photometric / astrometric calibrations
• Metadata
• image information from summit
• weather other ancillary summit data
• analysis statistics
• analysis history
• calibration information

All external data products are sent to PSPS for
external access. MOPS other science clients
also receive data products directly.
11
Additional Image Analysis Issues
Data has units of cell, chip, mosaic, image
group (major frame)

• Analysis is staged by data unit
• Phase 1 analysis preparation (mosaic)
• Phase 2 single image analysis (chip)
• Phase 3 calibration improvements (mosaic)
• Phase 4 image combinations (frame)

• Precision goals (PS-4)
• 30 milliarcsec relative astrometry
• 100 milliarcsec absolute astrometry
• 5 millimag relative photometry
• 10 millimag absolute photometry (internal system)

These goals can only be efficiently met after we
have produced a Pan-STARRS Astrometric and
Photometric reference catalog
12
Hardware Organization optimized for our parallel
processing
13
IPP System Architecture
state machine (not event driven) system
infrastructure is independent of
analysis architectural components are
stand-alone entities
14
IPP Software Architecture
processing scheduler
parallel processing controller
Software Tools CVS SWIG bugzilla
doxygen eups autoconf make gcc
processing scripts / analysis stages (phase 0-4,
cal 1-3, AstromRef, PhotomRef)
modules (debias, convolve, flatten, findobjects,
etc)
PSLib includes wrappers to externals (images,
vectors, errors, syscalls, etc)
External Systems (Lustre?, GFS?, Mysql, Apache,
etc)
External Libraries (glibc, glib, gsl, CFITSIO,
FFTW, SLALIB, etc)
15
Pan-STARRS 1

• primary PS-1 goals
• AP Survey
• verification surveys (MVP IVP)

• (especially for AP Survey)
• no simultaneous images
• little or no Phase 4 analysis (first year)
• no static sky
• limited reference data
• keep all raw data (1 year)
• multiple analysis passes

16
An Advertisement...

• We would like to hire 2 - 3 grad students for
  short-term
• IPP demo projects
• M31 variability analysis
• SDSS / Megacam proper motion study
• CFH12K / 2MASS i-band dropouts
• SkyProbe AE development and test