CASU meeting Wednesday January 26th 11:30-13:00 CASU meeting area. Present: JRL, MJI, STH, EGS, AKY, JPE, MR Apologies: JCR, NAW, RGM, SC Agenda: 1. Actions from last meeting 2. Comments on WFAU minutes 3. Hardware update 4. Meetings and telecons 5. Data archives update 6. Optical/NIR processing 7. WFCAM update 8. VISTA update 9. AOB Minutes ======= 1. Actions from the last meeting -------------------------------- STH - no final conclusions on the extinction dependence (though MJI and MJI STH have been chomping away at it) required before we can sign off the calibration for Version 1.1 of the data products. It's very hard to remove the residual scatter (0.1 mag) above E(B-V)=10. STH also <<<< reminded us of the VISTA-WFCAM issue (delta-mag vs mag) which we still don't understand. STH will check, once more, possibilities a la sky background estimation for next meeting - ongoing <<<< EGS - put the WFCAM illumination tables on the web MR STH - the WFCAM extinction calibration is closely linked to the VISTA one, MJI scatter for WFCAM in high E(B-V) improved but still - ongoing <<<< MJI - still needs to update the summary tables total exposure time column to account for occasional missing dither components, will do for v1.1 <<<< AKY - liaised with Henry from UltraVISTA and found that the artefacts around bright stars n'existe pas. A revised action is now to acquire an example of their latest stacks and examine the differences compared with our own - ongoing <<<< MJI - took advantage of a recent visit to ESO to lodge the idea that CASU processing of all the VST data would be beneficial to all. This would minimise disk phaffing at ESO (yes it will still come on USB disk apparently) and enable better processing of all VST data - ongoing <<<< STH - produced a first pass colour transformation predicting i-band mags MJI (for guide stars in SADT) from 2MASS JHKs photometry in time for tomorrows VISTA IOT telecon. STH showed some slides. Encouragingly it looks like we can predict i-band mags to within 0.1mag given JHKs. The next steps are to robustify the simple linear fits and characterise the extinction dependence - ongoing <<<< MJI - EGS and MJI went to the first day of the UKIDSS meeting at the RAS and MJI attended the VMC meeting in Munich. AKY - started to investigate the question of the evolution of the flatfields (and possibly darks) with time: looking at systematic noise/feature changes on different scales, and rms noise properties - ongoing <<<< JRL - the gas mark4 files were duly noted in the requisite processed subdirs EGS - continued investigation of grouting for VVV data aided by the VVV MJI folk contacting us about some odd spatial effects they were finding in a few tiles (saved us looking). To cut a long story short, to fix these problems required both grouting and also accurate individual detector zero-points to be used in the grouting process. The main culprit in these tests was the Z-band where average detector zero-point offsets of +/- ~1% were derived in this band for the month in question (Sep 2010). Possibly surprisingly, in crowded regions this is still sufficient to cause noticeable changes in spatial density of selected stellar pops from the CMDs. Anyway, at least after all this detective work, we know that grouting works even in crowded regions. 2. Comments on WFAU Minutes --------------------------- After spotting an apparent misunderstanding regarding versions in recent WFAU minutes, MJI sent a brief note clarifying what is currently VISTA v1.0 and then listing the upgrades that will happen for v1.1 as follows, plus a bit more detail: re-running the classifier on all pawprint catalogues; re-running the photometry on all pawprint catalogues; generating monthly detector zero-point updates (and illum corrs); FITS header updates on all pawprint catalogues/images; running the grouting code on the tile catalogues; followed by tile catalogue re-classification, re-photometry; and finally header updates on tile images or to summarise, no change to images, just header updates, some changes to catalogue table contents and headers. Why so many steps ............. classifier - improved estimates of aperture corrections (needed for grouter) and detector saturation levels (fixes occasional FUs) photometer - updated colour equations, and extinction coeffs and more careful selection of 2MASS stars in high extinction regions monthly ZP - detector level differences do matter even at +/-1% level and will be now routinely included for all VISTA data grouting - fixes both PSF changes via the various apercors and magzpt changes at either detector level or whole pawprint and also computes and adds in a column of per object delta MJDs + output of optional map of differential MJD across tile So the good news is we have added a MJD column for tile catalogues as part of the grouting process; the null news is that we decided to not write a TOTEXP header - NICOMB NDIT DIT suffices; the bad'ish news is that the saturation level estimates in v1.0 are occasionally rubbish (but in mitigation can be trapped using standard default estimates). The only piece of CASU pipeline processing that makes use of detector saturation level estimates is in the photometric calibration step to reject saturated 2MASS stars. Fortunately since this part uses the whole array to estimate a single zero-point this hasn't caused any noticeable problems. 3. Hardware Update ------------------ The Supermicro 45-bay JBOD expansion module est arrivee and racked. This should give an extra ~80TB of usable RAID6 storage when fully installed. The IoA power outage on Monday 20th December took us most of the day to recover from due to 5 separate power supply unit kauptskis. This used up all of our spares so we ordered 5 more. 4. Meetings and Telecons ------------------------ The VVV science meeting in December was held in Vina del Mar, Chile, though none of us could attend. JRL and EGS will have a browse of their website vvv.survey.org to see what's a foot. Related to this, MJI pointed out that the next VVV meeting will be a combined VVV+VPHAS+EGAPS meeting at Herts this July at which a few of us will no doubt turn up. <<<< MJI and EGS attended some of the UKIDSS science meeting at the RAS this January to catch up on what science is being done. JPE and MJI attended most of the recent VMC meeting in Munich. Lots of science deriving SFHs and analysing variables in the MCs was presented. MJI gave a CASU talk describing the processing and calibration and for WFAU NJC did a similar job on the VSA. One decision made during the meeting was that only deep stacked tiles and associated catalogues will be submitted for the March 31st ESO Phase III deadline, which, if it remains thus, lets CASU off that particular hook. The next VISTA IOT telecon is at 2pm UK time on 27th January (JRL, MJI, AKY will listen in). In advance of this STH has circulated to the IOT email list his investigation of infrared->optical photometry to predict i-band mags to aid in selecting more reliable VISTA guide stars. 5. Data archives update ------------------------ All going very nicely thank you. 6. Optical/NIR processing -------------------------- EGS noted a bit of a backlog with UVEX and IPHAS processing. JRL expressed his interest in the OmegaCam/VST header structure but it was felt prudent to await the union of the two before agitating for an example FITS file. STH has a large volume of CFH data on h and chi Per to process. MJI warned about illumination correction issues with CFH MegaCam data. Apparently the problem has been fixed recently - version B.5 illumination correction files used to make superflats prior to Elixir processing are worth looking out for in the FITS headers. Earlier versions (B.3 and B.4) leave a ~5% spatially varying residual photometric pattern. We are expecting to receive requests for updates to the VST survey management plans for ATLAS and VPHAS+ in the near future. The Kepler field imaging project got INT time and has a 10 night run towards the end of June. EGS mentioned possible updates for our in-house band merging software to include, say, more aperture fluxes across the bands. MJI noted that it may be more expedient to start from the current band-merged product format and make use of the pointer information therein to access any/or all of the parameters in the contributing catalogues. 7. WFCAM update ---------------- Processing and receipt of data is up to date, though there has been little data taken over the last month or so due to bad weather at the summit. During some of this bad weather ice formed on the cryostat window sometime after the 10th January. The effects, ~0.5 mag extinction in all bands plus enhanced K-band sky brightness, can be seen in data from 16th January through to 25th January, after which the remaining ice was cleaned off. Interestingly, other than the main effects noted above, as the ice was well away from the focal plane, it is hard to tell from the science product images that there was a problem. Data from 26th January is back to normal. Processed data up to the end of November has been checked etc.... and flagged as ready to go. Doing the monthly detector zero-point updates for December and January may be more challenging than usual and has been deferred until all the January data is processed in case we need to double up. There have been the usual few requests for early access to processed data and as expected no news from those lucky punters who requested access to the raw data. Finally we note that Watson Varricatt took over from Chris Davis a few months ago as the official contact person for all matters relating to WFCAM data processing. 8. VISTA update ---------------- We have received VISTA raw data up until 12th January, although 3 nights over the Christmas period were missed - the third disk was hiding in the bottom of the box, capisce ? So it was sent back, d'oh! A replacement has now been requested. Processing is now about to start with the January data. Both this and the December data will not be released until they are at version 1.1 level and until we have the missing December data included. Release-wise we have completed v1.0 up until the end of November. The updates to the catalogues required to go to v1.1 have begun on a duplicate copy. v1.0 of everything will continue to be available until all the catalogue v1.1 updates have been finalised. There will then be a brief haitus in access on a month-by-month basis while the v1.0 catalogues are replaced and the corresponding image headers updated. In addition to the differences between v1.0 and v1.1 noted earlier, v1.1 from December onwards will have slightly different astrometric distortion coefficients. Now that the telescope has settled through assorted focal plane adjustements we have (re)verified that the wavelength dependence of the distortion is negligible and confirmed that the minor tweaks noted several months ago are still applicable. The systematic residuals are already well below 25 mas (the requirement was 100 mas) and with the tweaks will drop even lower to ~10 mas. Other changes are related to keywords such as NICOMB, changing RADECSYS from FK5 to the correct ICRS (although a redundant Equinox keyword will still be present to ensure some backwards compatability). After a couple of independent requests a keyword recording the average on-sky PA of stellar images has also been added to the catalogues. Most of these changes are driven by trying to coerce decent results out of the tiles. There was some discussion of who is going to do what. As soon as the calibration updates are finalised MJI will start the ball rolling on the monthly detector zero-point computations and then hand over to the youngsters. <<<< In tracking down some of the problems with aperture corrections and saturation levels a couple of extra diagnostics have been developed which may be worth adding to the assorted checks. A few extra words about the problems with ESO OB status and grades have been added to the web pages with advice about using the derived QC info to help assess the science quality of the OBs. A good example is the 10% leeway on seeing from stacked pawprints allowed for "A" grading. The Paranal and Garching pipelines don't use drizzling in stacking, the Cambridge one does and generally drizzling leads to slightly worse measured estimates. This means an OB that was marginally "A" on seeing criteria may end up after being fully processed as marginally not "A". Tiles, yet more wholesale resampling and drizzling, compound this effect even more but don't necessarily impair, say, the limiting depth etc... which is a more subtle blend of seeing and noise characteristics [e.g. take any old image, blur slightly, measure seeing -> slightly worse, measure s:n -> exactly the same]. At days end it is the science assessment that counts. Regarding Phase III delivery our current take on this is: UV, Video - are both doing their own thing as far as we know VMC - are submitting deep stack tiles + catalogues, via WFAU VVV - is submitting tile images and catalogues via CASU VIKING - nobody knows VHS - are almost certainly submitting products from CASU any updated information on this gratefully received, particularly regarding plans to deliver v1.0 or v1.1. 9. CASUHELPs ------------- Ran out of time on this one and only had time to note how often questions about saturated star photometry keep arising -> LATFC + A. Which begs the two main questions, why ?? and why ?? AOB --- Flushing is believing, as is ever growing puddle of piddle = blue collar toilet ne marche pas, fortunately executive suite sigue funcionando MJI asked about support effort for CASUtools and how many downloaders there have been to date (23). So far not too bad but as it is deliberately not advertised too much we'll keep a weather eye on it. Continuing Actions: =================== STH - produce "final" VISTA colour equations extinction dependence and MJI finish revising calibration plan document MJI - finish WFCAM calibrations in regions of high extinction STH MJI - update summary table software for total exposure time mod for v1.1 AKY - liaise with H_Joy_M and acquire, if possible, example of deep stack for comparison MJI - contact KK about processing VST KIDS data STH - check extinction dependence and refine 2MASS JHKs -> i-band predictor AKY - continue flatfield properties investigations New Actions: ============ STH - check encore une fois possible causes of wfcam -v- vista mag slopes JRL - find out what's a foot at vvv.survey.org EGS MJI - carry on with v1.0 -> v1.1 process and report back at next meeting EGS AKY