7th December 2006 Hoyle Committee Room 10:30 - 12:30 Present: DWE, MJI, JRL, PSB, EGS, MR, RGM, NAW Apologies: WJS, STH Agenda ------ 1. Actions from last meeting 2. Comments on WFAU minutes 3. Report from UK Review 4. Data archives update 5. Optical processing update 6. WFCAM news 7. VISTA news 8. AOB Minutes ------- 1. Actions from last meeting JRL has decided to fix the problem with the UKIRT archive (missing header information problem) using a different approach, by modifying the file structure to create MEFs - ongoing <<<< STH has obtained a nice colour poster of the supernova remnant from an image taken by Albert Zijlstra. STH has not investigated the effects of stacking on the 2MASS/WFCAM non-linearity comparison - ongoing <<<< JRL is still awaiting a reply to his emails regarding the PostgreSQL problems with the AAT archive - ongoing <<<< It was noted that more AAT data will be arriving soon. This crate of DVDs will cover 3 years of data, so when ingested, the archive will be only ~one year behind observing. STH have finished tuning the colour equations and extinction term for Y MJI and Z (and JHK) using 2MASS photometry. The Z offset is consistent with 0.0 while the Y offset is between 0.0 and 0.1 depending on which dataset is used to measure it and how the intercept is interpreted. Consequently Y and Z offsets of 0.0 will be used for DR2. The main change for all bands is that an extinction correction term is applied based on the Schlegel maps. MJI have not emailed the new WFCAM colour equations to DWE and JRL STH - ongoing <<<< RGM has not finished working on the classification aspect of cross-talk weeding - ongoing <<<< JRL has added a detection filter FWHM choice as a command line option of the pipeline version of imcore DWE have not updated the Plone technical pages on astrometry. It had MJI previously been decided that when the astrometry gets updated, then the pages will be updated. However due to delays in updating the astrometry it was felt that perhaps this should be done sooner, with a clear indication of the current situation regarding processed data. DWE reminded everyone that this refers to passband dependance of the "r-cubed" distortion parameter PV2_3 - ongoing <<<< MJI emailed Derek Ives about WFCAM intrapixel sensitivity measures. The information is contained in Barron et al. (astro-ph 0611339). The bottom line is that this is a small effect for WFCAM. DWE said that he had looked at astrometric residuals as a function of pixel phase and that they were not detectable. EGS has backed up the ING archive. MJI noted that the tape changer works properly under linux (as opposed to solaris) DWE has been in touch with Phil Lucas about the PSF fitting comparisons. Three frames have been chosen to carry out the tests. MJI asked Mark Casali about curtaining and ESO IRACE controllers, who passed the ball to Gert Finger, who has not responded yet - ongoing <<<< JRL has released his report on VIRCAM AIT dark frames. DWE was asked to obtain the equivalent flat field report and release it. <<<< ALL have discussed their Gaia involvement with MJI and FvL EGS is in the process of organizing the purchase of a new server as spare for the CASU web services and for running IPHAS web service - ongoing <<<< 2. Comments on WFAU minutes RGM reported that all the 06A data has been copied over to WFAU and that the flat files are all available. There followed some discussion regarding what happens if a file gets altered in some way after it has been copied. MJI and MR noted that major changes eg. re-reduction of a night is flagged using a new version number, replacing the odd "broken file" is done on request (but with no flag updates), and said that they would rethink what to do for photometry/astrometry updates which are purely header updates <<<< PSB gave the latest news about the UKLight saga (established Oct 2005). There seems to be a sticking point with obtaining a piece of equipment at Edinburgh from UKERNA. RGM pointed out that this piece of kit was available from Amazon (300-500 pounds depending on model) with a few day delivery period. PSB noted that his efforts in contacting Sam Wilson and getting various tests started had gone unreported. RGM said that he could ask George Efstathiou to write a prodding letter to Andy Lawrence about UKLight to see if this would speed things up. 3. Report from UK Review MJI opened a brief discussion about the VDFS UK Review report by welcoming the generally positive tone of the report and thanking all concerned for their efforts toward this. PSB noted that no timescales nor personnel were specified for the loosely defined actions. The recommendation for another iteration of the UK URD was recognized as important but also contingent on taking note of revised aspirations in the 6 VISTA public survey proposals. Following on from the above, there is clearly an urgent need to (re)assess the revised VISTA public survey proposals for their data processing requirements. MJI proposed an internal meeting during the first week of January to discuss these in detail. <<<< RGM and MJI said that they would ensure the most up-to-date versions of the proposals were placed on the internal web pages <<<< Although there was still concern that the release of software would have support implications, no major problems were foreseen. JRL and DWE will sort out the release of the software, manuals and relevant documentation on the Plone site. <<<< One very positive outcome of the review was a clear statement about the need for the 6 month bridge in VDFS development funding. 4. Data archives update AAT: more data on DVDs is on its way. WFCAM: no data received yet for 06B. ING: a whole box of prehistoric DAT tapes has been found. These ~200 tapes probably only amount to about 1 Gbyte of data but are currently being ingested. MJI note that some other equally ancient DAT tapes, written using VMS backup, had been sent back to ING for reading. Our unix versions of VMS backup cannot cope with half-corrupted tapes. MJI asked about usage and feedback from the new ING archive interface. EGS noted that the main comment that he has received regarding the new automatic archive interface is regarding finding calibration files. This is now (currently at least) the responsibility of the end-user. A future upgrade path is to automate finding suitable calibration files as part of a longer term plan to provide on-the-fly reduction of these files. UKIRT: the problematic part of the archive is in the process of being fixed. JRL mentioned that the ING archive will be fully moved to Linux by the end of the year and the Sun Sybase server (cass38) will be decommissioned. 5. Optical processing update EGS said that IPHAS data was now available via AstroGrid. A tutorial on how to use AstroGrid has been written around the IPHAS data: http://www2.astrogrid.org/science/science-examples-stars/iphas-catalogue-images/ DWE said that he had installed the AstroGrid software for looking at the VO version of the Carlsberg catalogue and had no problem following the instructions: http://www2.astrogrid.org/science/getting-started/ Only one IPHAS 2-week run (containing 17 million objects) was uploaded. There are about 20 runs in total. 6. WFCAM news No data has been received yet. JRL said that tapes were reported to have been sent on 28th November. MJI noted that these are now LTO-II's. MJI listed some of the issues that have been raised by JACH about the 06B data: * the Data Acquisition system has been improved (again) * the first batch of data taken in NDR mode are in counts/s not counts * a zero countdown problem with the data acquisition has led to missing files and restarted MSBs * there have been periods when data from various detectors is not present * many MSBs have been interrupted * the flat field screen is shredding and will be removed. This was not foreseen to be a serious problem from the point-of-view of non-linearity measurements. MJI reported that he and Omar Almaini have been looking at the magnitude limits of the deep stacks in the UDS and DXS data. In particular correlated noise, present due to a variety of effects such as: inter-pixel capacitance; re-interpolation during deep stacking; unresolved background sources; low level sky residuals, directly affects the way magnitude limits are computed from the image noise properties. The correct way to deal with this is to either compute the 2D noise correlation function and use it to predict the magnitude limit (relatively painless) or to carry out quantitative object detection completeness tests on the real data (more painful). (Overlap tests eg. by splitting the data stacks in two, are also a quick way to get a reasonable assessment). The UKIRT board met recently and composed a gripe about the consequences of algorithm changes; namely that the PIs don't get a chance to check the effect of algorithm changes. MJI noted that he had replied pointing out the difficulty CASU had encountered in obtaining timely feedback on changes to the science products and that this was one of the reasons for pushing for rapid access to the flat file science products. STH's interim report on the Y and Z calibration is available at http://www.ast.cam.ac.uk/~sth/yz.html The final versions of the UKIDSS renewal proposals can be accessed via the UKIDSS wiki page: http://wiki.astrogrid.org/bin/view/UKIDSS/SurveyCall DWE said that the PSF fitting work was progressing. Test frames have been selected and processed. 7. VISTA news PSB reported on an ESO telecon held on 27 November with Pascal Ballester and Sandra Castro to discuss the upcoming 0.5 release of the software. The intention is to have a 0.4 release containing 10 out of the 12 intended recipes. This version is for ESO to test. After discussions during the ESO Calibration workshop in January, necessary fixes, enhancements etc.. will be made and the 0.5 version will then be released (target date end of February). This will be a major milestone release in the VDFS 1618 schedule. ESO created a version of the VISTA ETC during the summer and have been comparing outputs with the online version here. There are several differences but it quickly became apparent that most of these were due to: the old technical data tables that ESO have; the odd bug fix we have made; and their use of Gaussian rather than Moffat profiles. Consequently PSB has sent them updated versions of the ETC software, tables and documentation. The technical aspects of how the RECIPE keywords progress from user choice through to appearing in the FITS headers were also discussed. JRL reminded us of the VIRCAM dark frame report. The main feature is that there are large numbers (a few %) of hot pixels and related artefacts, the majority of which are well-behaved. Flatfielding seems to work as expected, with the (not unexpected) proviso that the flatfield must be taken using the same observing parameters as the target frames. It was noted that on detector 14 a whole channel seems to have intermittent bar-code-like noise. PSB noted that the RAL documentation set has been delivered, which was previously holding up camera shipment. RGM had paid a recent visit to the VLT and had visited VISTA. He reported that he had seen the telescope moving under computer control with his own eyes. He was told that the VISTA generators were now allowed to operate at night since there was no evidence that they produced a hot air plume affecting the VLT seeing (2 miles away). For a picture of the VISTA project team at the Paranal 2006 Halloween party, see http://www.flickr.com/photos/29433680@N00/285332566/ 8. AOB JRL said that he has been on another visit to London to sort out various ADASS 2007 issues. An additional hotel has been found which will give a nightly rate of about 30 pounds less than the primary one. Investigation of the wireless access at Kensington Town Hall reveals it to be enough for 12 people only. It will be possible to arrange with BT for additional temporary lines to be brought in (one needed per 100 users). PSB is in charge of this. Although there are a number of restaurants around Kensington High Street, the interval caterers will be providing a cash bar lunch service (at no cost to ADASS). Kensington Town Hall has enough poster boards, so these don't have to be hired. MJI said that he had recently been to a HSC-WFMOS meeting in Tokyo and reported on developments there. We were reminded that the CASU Christmas lunch will be on Friday 15th and that RWA is collecting the dosh. <<<< Continuing Actions ------------------ JRL finish the fix to problem with UKIRT archive (missing header information) STH investigate effects of stacking on 2MASS/WFCAM non-linearity comparison JRL repair AAT archive mirroring (PostgreSQL problems) MJI email new WFCAM colour equations to DWE and JRL STH RGM report back on classification aspect of cross-talk weeding DWE update Plone technical pages on astrometry MJI MJI if possible obtain information about curtaining in ESO IRACE controllers EGS organize purchase of new server as spare for CASU web services and for running IPHAS web service New Actions ----------- JRL release flat field frames report on Plone site DWE MJI consider how to flag header updates MR JRL release VIRCAM software and documentation on Plone site DWE RGM acquire most up-to-date versions of VISTA public survey proposals MJI organize internal meeting during the first week of January to discuss VISTA public survey data requirements ALL pay RWA for CASU Christmas meal