CASU/WFAU liaison meeting 09/12/02 --------------------------------- Pre-meeting: MJI,JRL,STH,DWE Summit Pipeline: Causal meaning only works on what's there now and not on what will come. This pipeline provides summit DQC feedback and first cut at science product. Standard Pipeline: Only run in Cambridge. Will produce the final science images and standard catalogues (but may need some tuning at commisisoning time due to lack of knowledge about various problems eg. PSF variations within sequences, stability of calibration frames etc..) Further Pipeline: PSF fitting must be after Standard. Perhaps provide later (split off task from Standard) alternatives -> use standard x,y and fit PSF flux; refine x,y and fit PSF flux CLEAN type approach full iterative solution Mike thought should use empircally derived PSF not some general function (eg. Moffat), empircal PSF must be oversampled and allow for x,y dependencies. New buzzword: Eigen PSF (Sloan). Advanced Pipeline: Cambridge should write a lot of the algorithms since already have several prototypes in place Issues: 16 bit integer or 32 bit reals: Can't see why 32 bits are needed. ? only if VERY short exposures and stacking or lots of NDRs ? Tests done on chips (one only): (a) Image persistence om (b) Crosstalk problem - but repeatable. Specify a 32x32 matrix and apply it. 0.1-1.0% at worst? One pass correction will be sufficient, ie. no iteration needed. If all 4 detectors cross-talk will need 128x128 matrix! (c) Reset anomaly - apparently repeatable. let's hope it stays that way. (d) Dark stability apparently ok. Test data: its proving difficult to get suitable test data eg. twilight & dome flats and WFCAM trial data. We know what we need, but can we get it? The rest of the detectors are expected in February. Extra parameters: Extra apertures - Kron, Petrosian, FWHM - no problem. (Do we want elliptical apertures for the Kron magnitudes? - apparently not (SJW). Proposal: 12 apertures should be good enough. We will then additionally measure the Kron, Petrosian, FWHM radii. Could get those magnitudes from the 12 "standard" aperture magnitudes (if grid is fine enough) but may as well derive them as extras to stop future moaning. Processing flags - set bits in 32bit word eg. edge (and other problems) cf. SDSS error bits Errors - x, y, fluxes, but not necessarily ellipticity & areal profiles. (Nigel Hambly and Ian Bond arrive after lunch) revisit a few earlier issues ...... Discussion about fluxes/apertures etc. STH says put Kron and Petrosian magnitudes in, so as to shut people up. Also suggested vignettes (5x5 pixels or whatever). After some discussion decided no point in these. Why 32 bit data product? ~10 bits of dynamic range so 16 bits should work. No clear answer why, except for if integrating within the DAS (Data Acquisition System) - which is not going to be done. This should halve the data rate. The "standard" (well for LAS at least) exposure of 10s may now be 2x5s with a shift of n pixels between the exposures. This is in order to correct for bad pixels (~1-10% of them!). This will double the data rate if use 16 bits its back to where it was again. (SJW joins meeting) to recap ...... Profile fitting: 1) Profile fit without centroid adjustment. 2) Profile fit with centroid adjustment. 3) DAOPHOT/Clean algorithm (Full Monty) - multiple sources. SJW: generic (galaxy) profile fitting possibly not needed. Talked to Sloan and even though these were provided, they haven't been used. Profile (exponential?) fitted magnitudes (for galaxies) - possibly not useful for photometry, but sharpness measure could be useful for classification etc.. SJW suggested that we say we'll only do stellar PSF fitting and see if anyone screams. Sextractor parameters: Stellaricity? APM classification parameter better. MJI: use Sloan list rather than Sextractor's list for ideas. SJW: suggest galaxy profile fitting uses exponential and de Vaucouleur profiles (does not tally with previous stement above). SJW said elliptical apertures (for Kron magnitudes) not needed. 12 apertures suggested by MJI. Sample (?) radii. SJW wanted Sloan radii. MJI said that this was too coarse. The current APM system is radii every x sqrt(2), but this is probably also too coarse and suggested area every x sqrt(2). Saw no point in going as far out as Sloan. SJW suggested that we at least match some of Sloan's radii (but why would this be useful? different seeing, PSF etc.). [For suggested set see new parameter list on diary web page] Simulations needed perhaps - get someone keen to do them. How much does seeing affect matching Petrosian (etc) magnitudes between Sloan and us ? How good are derived parameters, errors, image detection, completeness esitmation usw.. Advanced products: much discussion Change name to database-driven products in order to clarify? things. List driven photometry: when new passband is ingested by the database, check the other passbands already taken for the flux at that point. CASU write the software, WFAU run it. Stacking: WGs should define a limited number of stacking rules - what do they want? Also, we need to say what we can provide and by when. What is a default stack? How often should stacks be updated ? Photometric calibration: Will definition of the standard fields be done during commissioning? (based around UKIRT faint standards?) SJW will write a document that will state that this will happen and how it should be done. MJI thought it was 4 months commissioning followed directly by science. SJW thought it was more like weeks. SJW will check to see what's intended, then we can comment on it. Skyprobe: Generally thought of as a good idea. SJW is going to write a document to justify using it to Andy Adamson (timescale 3 months). Can you go from V to JHK? Would r' be better given we have to calibrate it from all-sky photomatry eg. TYCHO, therefore may not want to go to i' and beyond? STH has used it and said it was great. Skyprobe data (1 year's worth) is available from the web (to check out sky conditions - who is going to analyse it ? eg. percentage of photometric time etc..) Network connectivity: Is the ROE firewall a problem? IoA has no specific firewall - Gbit line. MJI suggested a test. IAB said that copying 1 Gbyte of data took an hour. A bottleneck does exist. NCH wanted to play with some CIRSI data. MJI will check with JRL (plus wants some software modules/code). FITS headers: Any news? WAFU need updating. 16 bit (again): DAS was originally envisioned as doing a lot of co-adding and therefore needed 32 bit. This doesn't seem to be the plan any more. MJI will ask Mark Casali. Went over Science Archive requirements document: Catalogue ingest speed Database integrity (backup exists) Calibration version control Persistence of queries (?) SJW: Advanced products documents? Should now be called database driven and will be folded into CASU/WFAU documents. Subaru tasks: would be nice to have some, but can't be anything vital. MJI said that investigating the general stacking problem has already been suggested. Acceptance tests for pipeline? Good idea, but how to specify it. Should be mainly done from simulations since there's no point doing it if you don't know the answer in the first place. SJW will look into simulations (by 1 Aug - something to do with one month before acceptance tests needed?) and also as extra sanity check a comparison of UIST/UFTI data processed by SJW by hand cf. pipeline.