Notes from UKIDSS meeting 25th November 2002
Updated after CASU-WFAU liaison meeting 9th December 2002 (+NCH, IB and SJW)
Overall philosophy
------------------
Summit - causal pipeline: DQC feedback, first pass science products
Standard - nightly pipeline: DQC check, astrometric and initial
photometric calibration, standard catalogue parameters,
final 2D science images
Further - nightly pipeline: PSF estimation and general profile fitting
Database - asynchronous processing: stacking, difference imaging,
driven temporal, matching and colocated list driven stuff
(inc. external list drivers), contiguous tiled regions and
seamless overlap calibrated catalogues, optimal detection and
parameterisation of eg. LSBs (ie. bigger than PSFs)
(note basic --> standard since not such negative connotations and
advanced --> Database driven since some of the processing not that
advanced per se)
2D instrumental signature removal
---------------------------------
effects to deal with may include: persistence (if present), cross-talk,
dark stability and non-linearity, flatfield stability, defringing,
sky variation, telescope/dust thermal emission
non-linearity, reset-correction (de-biassing) dealt with in data
aquisition system
Issues:
variable seeing on short timescales, high proportion of bad pixels
should also consider stacking cf. interleaving for shallow surveys ?
may need to do PSF fitting on individual components of interleave
irrespective of above
for shallow surveys can we use 16 bit unsigned products rather than -32 bit
reals ? can do for much of pipeline anyway
test data:
need to aquire test data from WFCAM to assess persistence, reset anomalies,
dark stability, flat stability, bad pixel problems etc... asap
what other UKIRT?WFCAM test data do we need ? - more flats (dome cf.
twilight sky cf. dark sky) dark stability, WFCAM test data - cross-talk
image persistence, reset anomaly, dark stability
Standard 2D processing at summit and in Cambridge to be ready for Version 1.
with some tuning during commissioning and science verification.
Standard products
-----------------
additional image parameters:
Kron radius: r_k equal to 1st moment of radial distribution
Petrosian radius: r_p defined in terms of annular surface brightness
parameter \eta
Kron flux: summation out to k*r_k can be elliptical or circular ?
(similar to the total flux which it could replace)
Petrosian flux: summation out to k*r_p only seen circular, is elliptical
worth having ?
FWHM alternative: image FWHM summation out to k*FWHM, since for standard
profiles is at least as good as above two
finely? sampled circular aperture fluxes of fixed sizes - similar to what
we already have except more
suggest: Pk, core/2, core, sqrt(2)*core, 2*core, 2*sqrt(2)*core, 4*core,
4*sqrt(2)*core, 8*core, 8*sqrt(2)*core, 16*core, 16*sqrt(2)*core, 32*core)
where core = ~ for stellar images for site
Note that if we set core = 0.5 arcsec then the above sequence leaving out
Pk, would read apertures of diameter:
0.5, 1.0, 1.4, 2.0, 2.8, 4.0, 5.7, 8.0, 11.3, 16.0, 22.6, 32.0 arcsec
[see Staunton etal. p 498 for SDSS apertures - problem with them is they
are too coarse a grid (eg. x2.5 steps in radius) and extend to far too
large a radius given the likely 2D processing constraints on WFCAM data]
Note that for circular apertures all of Kron, Petrosian and FWHM are all
subsumed within circular fluxes if interpolate from fine enough grid as
are SDSS aperture (different seeing etc.. implies will need to interpolate/
correct to really match the SDSS ones anyway)
how fine ? what spacing ? how big a radius ? above are suggested set.
finer grid means can always pick optimal aperture for stellar images
aperture corrections for stellar images are useful adjunct (already computed)
Note that standard background estimation ultimately defines max. radius
as may 2D processing and is expected to be <~30 arcsec.
couldn't see anything extra in Sextractor list worth having not covered
in above and current lists
image morphological classification: is spurion, stellar, non-stellar enough
together with the pseudo N(0,1) statistic ?, could use Bayesian priors
when calibrated + other knowledge eg. globular cluster..........
profile fit results are useful extra information for image classification
need to add in errors on certain parameters and processing flag a la
SDSS one eg. set bit for hit edge etc...
Standard products to be ready for Version 1. with some tuning during
commissioning and science verification.
Further products
----------------
PSF generation methods, oversampled spatially varying PSFs good news,
how to form them cf. SDSS, HST, 2MASS ?
may need to use components of interleave ?
oversampled real PSFs vital for correct fitting of undersampled images
Options for PSF fitting are:
1) Profile fit without centroid adjustment using standard multiple components
2) Profile fit with centroid adjustment using standard multiple components
3) DAOPHOT/CLEAN algorithm (Full Monty) - multiple sources with full iterative
solution, adding and removing components as necessary (CPU intensive) and
obviously won't work for galaxies ie. prbably useless for generic higher
latutidue fields
do we additionally need more general profile fits eg. de Vaucouleur,
exponential, Plummer ???
Aligning with DB versioning: further processing Version 1. would include
options 1,2 for PSF work. Option 3 would be developed for Version 2.
Database-driven products
------------------------
Organised by DB but run using general purpose selectable agorithmic toolkit
asynchronous operations such as
list driven photometry, vital to make same measurements across all bands
master list created how? from union of catalogues in each band ? + external
lists ? stacking images first then generating catalogue in general not so
good since s:n will be worse for interesting images, should be RA, Dec list
list drives software processing via WCS, can use object masks to improve
standard background estimation. External (no-WFCAM) lists can also be
incorporated.
Version 1. just WFCAM data
Version 2. externally derived lists and cross-photometry on external datasets
stacking, complex problem, not necessarily single optimum solution eg.
image detection cf. resolution of higher s:n images, Subaru involvement,
JPE's e-science project, loads of possible methods, inc. re-running
image detection alogrithms
Version 1. stacker and mosaicer, straighforward algorithms with ability to
deal correctly with, scaling, confidence maps, spurion rejection, and
accurate WCS alignment
Version 2. optimal use of varying PSF information for image detection
problem and for image resolution problem.
Version 3. complete image restoration using all data contemporaneoulsy
LSBs, or more generic image detection methodology, not discussed at
UKIDSS meeting, is it worth pursuing for NIR arrays ?
Version 1. re-detections using fixed large detection filters having
previously removed standard image detections. Assess feasibility of
LSB work in NIR mosaic data.
Version 2. await results for Version 1. before proceeding.
detection of transient phenomena, adpative kernel matching, difference
imaging continuum subtraction, inc. re-running image detection alogrithms
Version 1. ability to do continuum subtraction for narrow band (eg. H2) work
Version 2. the rest.
--------------------------------------------------------------------------
Timescales for Versions 1-3 are:
Version 1. ready for WFCAM commissioning eg. last quarter of 2003
Version 2. one year after survey operations begin eg. notionally end of 2004
Version 3. rollout sometime later though development can be concurrent
--------------------------------------------------------------------------
Other Issues
------------
Setting up standard photometric calibration fields regularly spaced in RA
need to ensure this gets done early on and that standards in all broadbands
used in a night are taken at regular (1hr?) intervals. STH to coordinate.
Skyprobe local extinction monitor -v- ad hoc mishmash of AG measures etc..
MJI, DWE, STH, JRL
-------------------------------------------------------------------------
ACTIONS
-------
1. CASU to produce updated CDR document taking account of recent
discussions - needs timeline for various components ONGOING
2. WFAU to produce updated document ditto, both by end of
December ONGOING
3. MJI to contact MMC about acquiring recent WFCAM test data,
(received test data and report) DONE
4. STH Dome flats -v- twilight flats -v- dark sky flats, test data
still needed, STH to try again in December - nada. AA to
attempt with UFTI on 15th January ONGOING
5. JRL to get prototype WFCAM FITS header for images and forward to
WFAU for comments ONGOING
6. JRL to put together example data and catalogues from CIRSI data
for use in tests of ingestion, stacking/mosaicing at WFAU ONGOING
7. SJW to contact MMC about commissioning duration and plans,
particularly with regard to setting up photometric standard fields ONGOING
8. STH assuming CASU processes the data who is going to do colour
equations produce the standards and (keep updating them) do meso STH
slew tests - STH??? +??? decide VOLUNTEERED
9. MJI to ask MMC about plans for DAS 5s exposures, 16 bit alternative
data format DEBATING
10. SJW to set up simulations to test parameter generation software,
sensitivity to seeing etc... ONGOING
11. SJW to provide/select a night of UFTI/UIST data to compare 2D
processing ONGOING
12. WFAU to do more tests to find bottleneck in network connectivity
between Cambridge and Edinburgh. Current large scale transfer
achieved speed at 2.4 Mbit/s is hopeless ONGOING
13. CASU to forward prototype stacking and mosaicing software to WFAU for
tests. DONE
Last modified: Thu Jan 23 14:57:27 2003