Pipeline processing
The processing operations listed below have been applied to the
reduced data using IRAF for the overall structure, but with
supplementary extra stages as indicated. Most reduced data frames
have been converted back to unsigned two byte integer format to save
on transfer and long term storage requirements, however, reduced short
exposure photometric standard frames are stored in four byte real
format due to the presence of -ve pixel intensity values.
- De-biassing and trimming. Stacked bias frames using
the default clear (flash) and readout (slow) speeds show some
low level repeatable structure therefore full 2-D bias removal
was necessary. For debiassing and trimming the data and bias
sections as specified in the FITS header were used.
- Bad pixel replacement. Bad pixels and dud partial
columns have been flagged and interpolated over. The bad pixel
mask files are available here: [CCD#1]
- [CCD#2]
- [CCD#3] -
[CCD#4] (note that CCD#2
has zero entries in the bad pixel file) The SDSU controller
upgrade resulted in a slightly different data format
ie. changes in the overscan and underscan regions -and the
relevant upgraded bad pixel masks, post 31 August 1999, are
here: [CCD#1] - [CCD#2] - [CCD#3] - [CCD#4]
- Non-linearity correction pre-Aug 1999. Linearity tests
using sequences of dome flats revealed CCD#2 and CCD#4 to have
significant non-linearities over the whole dynamic range.
CCD#1 and CCD#3 are essentially linear to <1% over the full
range. A non-linearity correction has been applied via a Look
Up Table (LUT) to all data. CCDs #2 & #4 share one ADC and #1
& #3 another. We believe most of the non-linearity arises in
the ADC and should therefore be stable with time (see known WFC foibles for more
details).
- Non-linearity correction post-Aug 1999. After the SDSU
controller upgrade the non-linearity characteristics changed.
Now all CCDs are non-linear and need LUT correction. We are
still monitoring the stability of the new system and now have
reliable measures for September, October 1999, and March, June
2000. The pipeline processing highlights changes in the
non-linearity and we also this to monitor the stability of it
for each WFS run (see known WFC foibles
for more details).
- Flatfielding. For each observing run a sequence of
master sky flats was constructed. Because of readout overheads
it is impossible to generate complete sky flats in UBVRIZ each
night, hence flats from several nights were combined to produce
the master flats. Some low level fringing <1% is still visible
on the master I and Z sky flats but for the current processing
we have ignored this problem (see the section on flatfield frames for examples).
- Defringing. Both I and Z-band data frames suffer from
significant sky fringes. R-band images also show weak fringing
due to the red tail of the Harris glass filter. We are
currently investigating and developing algorithms for
automatically removing the sky fringes and are updating the
reduced data accordingly when we are satisfied with the results
(see the section on fringe frames
for examples).
- Gain correction. Although processed independently a
mean flatfield sky level was used to place all the CCDs on the
same response zeropoint system, ie. sky in deep frames should
be the same for all CCDs to better than 0.5%, the residual
non-linearity level.
- Astrometry. An astrometric solution based on GSC
objects visible in the data frames has been used to define a
1st pass astrometric solution. World Coordinate System (WCS)
information is embedded in the reduced images FITS headers.
Currently the external accuracy of this astrometry is limited
by the GSC accuracy to about 1 arcsec. Further improvements of
this are planned. Note that most extant image display programs
do not recognise the RA--ZPX and DEC--ZPX keywords and an
alternative interim system such as RA---ZPN and DEC--ZPN can
used instead (see astrometric
distortions for more information).
- Photometry. Approximate photometry at the 10% level can
be obtained by using the following default CCD zero-points: U =
23.6, B = 25.7, g' = 26.1, V = 25.6, r' = 25.6, R = 25.6, i'=
24.9, I = 24.9, z'= 23.4 ie. that magnitude star which gives 1
photon/s total. Note that these zero-points are in photons and
the Z' system denotes natural WFC filter +CCD response
approximately normalised to the Vega system.
The renormalised CCD gain is approximately 3.12 for pre-August 1999
data and approximately 2.6: September 1999 through December 1999
post-mscred taking over the flatfield gain renormalisation. However,
the gain and non-linearity of CCD#1 was changed sometime between the
December 1999 and March 2000 runs, such that the average gain (ie.
effectively what mscred uses) is now 2.75:, therefore to convert above
to counts/s magnitude zero points subtract 1.24, 1.05: and 1.09:
magnitudes respectively. Full colour equations and accurate
photometric zero points plus extinction correction information for
each survey run are available in the
the section on photometric calibration .
- Object Catalogues. A standard APM-style object
detection and parameterisation algorithm is being run on all
data frames to provide a first pass image catalogue for the
data. The output format for the catalogues is a separate FITS
binary table. See the release pages
for a detailed explanation of the catalogues. A cookbook gives some suggestions for
using the catalogues.
For further details contact Jim Lewis or Mike Irwin
( jrl@ast.cam.ac.uk or mike@ast.cam.ac.uk )