IRDR (CIRSI) Data Reduction Manual
Version 0.1, Jan 2000

Outline:

I. Review of standard data reduction procedure for IR imaging

II. CIRSI data reduction tricks

----

I. Review of standard data reduction procedure for IR imaging

The fairly standard data reduction procedure for processing near-IR
imaging data is a two pass reduction using running sky frame subtraction
and object masks from the first pass coadded dither sets.  Examples
of software packages are PHIIR and DIMSUM.  Some references include:

Hall, Green, and Cohen 1998 ApJSS 119 1
Stanford, Eisenhardt, and Dickinson 1995 ApJ 450 512
Thompson et al. 1999 AJ 117 17
McLeod, Bernstein, and Rieke 1995 ApJ 96 117
Cowie et al. 1990, ApJ 360 L1

The procedure is roughly:
  1) Correct data for non-linearity
  2) Subtract average dark frame
  3) Running sky subtraction and flatfielding (order can be switched)
       Form median image of N neighboring frames (not including current frame)
       Normalize to get flatfield, and divide data frame by flatfield
       Form median image of N neighboring frames (not including current frame)
       Subtract from current frame
  4) Find dither offsets
       Detect objects
       Match object lists to calculate translation between dither frames
  5) First pass coadd
       Shift images to register
       Coadd and produce exposure map
  6) Create object masks
       Detect objects in coadded dither set
       Unregister object detection to create a mask for each data frame
  7) Running sky subtraction and flatfielding again, with object masking
  8) Second pass coadd

----

II. CIRSI data reduction tricks

The reset (bias) level can vary with loop number, including hot pixels
and structure.  Processing each loop separately (instead of coadding loops
as the first data reduction step) improves the resulting image quality.

The dominant residual structure due to reset problems is a dark band
extending over about 10 or 20 rows.  Subtracting the row mode from
each row, working quadrant by quadrant, removed this structure well.
Using the same correction on all frames (eg, from the median frame)
seems to work less well (reset artifacts should probably not be assumed
to be time constant).

Chip 3 has a boomerang type structure that is time dependent.  Using a
running sky frame subtraction of narrow half width (say 2 frames) helps
remove this effect.  (Chip 3 has been replaced but the old data still
need to be reduced).

Registering the dither frames within a dither set can be difficult to
automate with ~100% success rate because there are sometimes very few
objects in a given frame (especially if data was dropped so there is only
1 loop available).  By thresholding the images (setting pixels below
an object detection threshold to 0), and cross-correlating two dither
frames, the translation between dither frames was found with about 100%
success rate (at least on the NTT field).

Growing object masks (obtained from combined dither sets) by a
multiplicative factor works better than the usual technique of growing
the masks by N pixels.  That is, the growth should be proportional to
the object size, doubling the object detection isophote region to produce
the object mask seems to work well.

Image shifting using a variation of bi-linear interpolation that accounts
for bad pixels.  Output image pixel is obtained from weighted average of
the four overlapping input pixels, discarding bad pixels.  The weights are
the area overlap of the output pixel with each of the four input pixels.
Even when overlap bad pixels, the output mean signal level is constant.
The bad pixels are accounted for by shifting the weight maps using a
weighted sum instead of average.

The calculated dither offsets should probably be averaged over all chips
for each run.  The CD matrix (FITS WCS) should probably be averaged over
all runs for each chip.
