5th January 2006 Hoyle committee room 10.00 - 13:00 Present: DWE, MJI, PSB, STH, MR, JRL, RGM Apologies: NAW, EGS, WJS Agenda ------ 1. Actions from last meeting 2. Comments on WFAU minutes 3. UKIDSS SV meeting 4. VDMT meeting 5. VISTA ESO public survey meeting 6. VDFS VISTA update - telescope status - VIRCAM Status - AIT - VDFS docs status - DRL status - ETC status/workpackage issues - Template issues - twilight, njitter protocol 7. WFCAM processing - photometric calibration status - recent changes to pipeline - progress on PSF fitting - data reduction progress 8. Data archives update - AAT, ING, WFCAM, UKIRT 9. Optical processing update - VST, IPHAS, 10. AstroGrid liaison update 11. AOB. - GAIA - WFMOS Minutes ------- 1. Actions from last meeting WJS has passed on the SADT comments to Martin Folger JRL has not finished off the OMP database access scripts - ongoing <<<< JRL has not added an MSB quality flag to the FITS header - ongoing <<<< MJI has discussed the flatfield and sky properties draft report with MR with the conclusion that a paper will be based on it MJI continues to investigate the merged classification statistics - ongoing <<<< WJS was not present so it is unclear whether visiting RAL during AIT3a is still a possibility - ongoing <<<< PSB has discussed UKLight with Peredur Williams and there is nothing significant to report WJS provided PSB with updated measurements of the VISTA filter characteristics EGS distributed a set of options for the group web pages and are collating DWE the results. An offline meeting was suggested to discuss further progress. DWE added that the VDFS web pages now have a search facility MJI has produced a diagram to explain the cross-talk pattern. This has been added to the SV report (http://www.ast.cam.ac.uk/vdfs/docs/reports/sv) STH has recommended (in conjunction with the calibration working group) a new strategy for observing standards. 2MASS can be used to calibrate JHK to better than 2% (tested against UKIRT FS). Issues for Y and Z have not been fully resolved yet. Observing of standard fields has been recommended to be every 2 hours on photometric nights only. Among other things this will provide useful diagnostic measures and external verification. The report is available at http://www.ast.cam.ac.uk/vdfs/docs/reports/2masscal.pdf STH said that he was considering attending a meeting on photometric, spectrophotometric and polarimetric standardization in Belgium (8-11 May 2006 http://www.vub.ac.be/STER/standards/stds.html) STH did not select a more crowded field in which to carry out further photometric tests. It was by-passed by the decision to use 2MASS to investigate and measure the large-scale systematics in the photometry. DWE tested circular apertures for the PSF work and found no improvement due the symmetry of the modified Gaussian polynomial functions being used. DWE generated some colour-colour plots from a crowded field (M17) to test the PSF-v-pipeline photometry (see item 7) 2. Comments on WFAU minutes It was noted that the PI data has been released and this is welcomed by CASU. The "missing" data from 17 April was discussed. The OK_TO_COPY file is dated 7th October along with all the other SV data, so it is unclear what exactly happened. RGM raised the issue of data transfer. He had noted with help from PSB that 2 TB of data had been transfered in a few days around 19 December. Although this was 40 nights of data only 1 night was logged as having been successfully copied. RGM had been in comunication with Nigel Hambly (WFAU) and Nigel had explained that there had been a power-cut at the WFAU end and the copy had been aborted. The copy status would be updated soon. [Late breaking news; as of Mon, 9th Dec, the copy status now showed all data upto 29th Sep had been successfully copied to WFAU.] RGM stressed that the important thing was that there is no problem with the transfer rate and that therefore the current 2 month backlog in the transfer of processed data could be resolved with a few days of sustained transfer. As of 6th, Jan, all WFCAM data upto Dec 10th Dec (apart from 3 nights) had been processed and checked by CASU as available for WFAU ingest. RGM noted that this was an impressive achievement to start the New Year with. STH asked whether the issue of inconsistent images attributes mentioned in recent emails had been sorted out. MJI noted that this was being investigated and was due to incorrect information in the raw frame FITS headers for a 2 week period during May. JRL had manually fixed most of these at the time of processing the 05A data but a few may have sneaked through. 3. UKIDSS SV meeting MJI reported on the UKIDSS SV meeting held in Edinburgh on 15 December. See http://wiki.astrogrid.org/bin/view/UKIDSS/SciVersEdinDec for the minutes. The main topics discussed were the release policy and how to make the best of the processed data (see below). It was noted that stacked DXS data and catalogues need to be generated by the end of January. EGS is doing this work. <<<< The UDS team have requested that some processed 05B data be released by then too since no 05A UDS data is available. The status of this request is still unclear. RGM reported that most of the problems arose from people using the wrong parameters or not tuning the matching criteria appropriately. For example, Hall magnitudes were still being used by the GPS and GCS SV teams, even in very crowded regions, rather than the recommended aperture magnitude. Furthermore a matching radius of 5" was being used, which naturally gives rise to many outliers in crowded regions. MJI demonstrated at the meeting how good the catalog data is if you use the correct procedure. RGM has been investigating star-galaxy classification by comparing the pipeline results with respect to Sloan photometry and spectroscopy. The end result is that they are very comparable when taking into account the different noise levels. At the meeting Steve Warren raised the issue of Moon reflections and various ghost images. In order to help users assess if they are getting the most out of the data, MJI said that he would place example colour-colour and colour-magnitude diagrams for IC4665 and M17, together with a brief howto, in the SV report.<<<< 4. VDMT meeting MJI noted that one of the topics discussed at the last VDMT meeting (16 December) was the new requirement for a live up to date on-line ETC. This was discussed later (see item 6). The next VDMT meeting will be in February. 5. VISTA ESO public surveys meeting STH and RGM attended this meeting (20 December at QMW) which was to start the discussion of the ESO public surveys on VISTA. RGM was concerned to note that the VISTA Z filter had not been ordered yet. Individual coordinators (below) were designated to coordinate further plans and discussion on potential VISTA public surveys to prepare for the further meeting in the first week of February. Please contact them to join the discussions and flesh out draft proposals. Overall Jim Emerson j.p.emerson@qmul.ac.uk to coordinates the plans below and others that are proposed Hemisphere Survey Andy Lawrence a.lawrence@roe.ac.uk this includes the VISTA counterpart to the VST ATLAS survey Galactic Plane Phil Lucas pwl@star.herts.ac.uk this includes the VISTA counterpart to the VST VPHAS+ survey IR - 1000 sq deg Will Sutherland wjs@ast.cam.ac.uk this is the VISTA counterpart to the VST KIDS survey IR - 5k Sarah Bridle sarah@zuaxp0.star.ucl.ac.uk this is the VISTA counterpart to the proposed CTIO Dark Energy Survey this may give way to the IR-1000 sq deg survey Magellanic Clouds Maria Rosa Cioni mrc@roe.ac.uk Deep - 100 sq deg Gavin Dalton g.dalton@physics.oxford.ac.uk The milestones for the public surveys are as follows: 15 Jan 2006 Announcement 15 Mar 2006 Deadline Jun 2006 ESO Public Surveys initial selection and meet with PIs Oct 2006 Final proposals to OPC Nov 2006 OPC decide on proposals Apr 2007 VISTA commissioning and acceptance May 2007 Science Verification May 2007 ESO Public Surveys start Jun 2007 Call for Proposals for other surveys Sep 2007 ESO open time starts The next meeting will be end of January or beginning of February probably at RAL. 6. VDFS VISTA update - telescope status No WJS, so no update - VIRCAM Status - AIT PSB said that AIT3 had happened and that he had obtained about 50 dark frames (which had some light leaks). These were very useful since they helped clarify the orientation of the chips which was different to what was originally thought. A particular problem is the ordering of the extensions within the FITS files which is essentially random. The data is read out in blocks of 4 detectors and each block is stored on a first-come first-served basis and routed like that to the ESO archive. Not only does this look very unprofessional, it will cause programming problems in the future. It was felt most strongly that this situation should be corrected at source. MJI noted that the ESO WFI MEFs do not have this problem and said that he would ask Konrad Kuijken what the situation is with the OmegaCam data acquisition. <<<< - VDFS docs status PSB reported that there is a bit of a backlog with signing off these documents, although they are not waiting on any CASU action as far as we know. Some modifications have been carried out in order to be compatible for the DRL release v0.1. During the last 2 weeks of December three documents have been sent off for approval. The "DFS Impact", "Calibration Plan" and "DRL Design" are now all awaiting signatures before they can be released. It was felt important that these documents get to ESO soon. There was some discussion as to when the next deliverable of the DRL (v1.0) is. The last GANTT chart we have says 24 March, but it is unclear if this is the current target date. This needs clarifying. <<<< - DRL status JRL reported that he had sent off an unofficial release of V0.1 of the DRL to ESO which contains 4 recipes extensively tested (to destruction) by PSB. - ETC status/workpackage issues A new work package seems to have been created by the VISTA overlords, namely a live up-to-date fully functioning web-based ETC provided by CASU. The list of demands is quite large and will take a considerable amount of work to implement and test. There was some discussion as to what part of the VDFS project should be deferred in order to free up some manpower effort to undertake this task. - Template issues - twilight, njitter protocol PSB and MJI noted a slight problem with the twilight template design in that it lacked an offset/jitter pattern to make the twilight flats useful. MJI, JRL and PSB will put their heads together and suggest a suitable strategy. <<<< There was some discussion regarding the level within a flat and real-time determination. 7. WFCAM processing - photometric calibration status STH noted that the Photometric Calibration document has been revised and now sent off as a recommendation. This covers J, H and K. For Y and Z the plans are to further assess 2MASS using a linear colour term (as is currently being used) by comparing the Z-band calibration with SDSS. The crucial proviso is to use a colour limit (eg. J-K < 1) so that the population density differences between dwarfs and giants and extremely reddened stars are not a problem. - recent changes to pipeline JRL reported that the pipeline is stable at the moment. Since the last meeting the biggest change is an upgrade to the determination of the skies. All available suitable frames will used to determine a "local" sky frame to effectively completely eliminate extra contributions from rms noise. A few minor alterations/bug fixes for the summit pipeline requested by JAC have also been implemented. - progress on PSF fitting DWE reported on PSF fitting, the main progress having been on robustifying the measurement of the PSFs. A report on the latest results has been written and will be released soon. <<<< Data covering M17 was analysed and colour-colour plots generated in order to compare the performance of the PSF and standard pipeline photometry. For the frames with the best seeing, there was not much difference between the two methods. However, for worse seeing it was noticed that the pipeline performed better. The reason for this was that the PSF was changing between the microsteps and thus a standard PSF model was not appropriate. The astrometric accuracy was not affected by this and similar performances was obtained for both methods. In order to cope with this situation a new PSF determination method is being developed in which the microstep interleave components are analysed in parallel. Work on this is still in progress and has delayed incorporating PSF measures into the catalogues - though in lieu of the earlier comment this is hardly science-mission critical. - data reduction progress Tapes of WFCAM data for the period up to the 12th of December were received on the 21st December. These were ingested over the holiday period and processing of earlier data continued over the supposed Christmas break. CASU are currently processing data up to mid-December and have completed processing all data up to the end of November ie. as planned, on average about a month behind data being taken at the telescope. For an up-to-date view of the processing status see http://apm15.ast.cam.ac.uk/wfcam/ report_night_reduction_status?semester=05B&SUBMIT=Submit+Query, also accessible via the vdfs front page. The state of processing on 6th Jan, 2005 was: data up to Dec 17th was either processed and checked or is being processed data up to 22nd Dec has been verified and ingested 8. Data archives update - AAT MJI asked about progress with the AAT archive. JRL noted there were still one or two minor changes to make. JRL also said that he would ask the AAT if there is any more data that is available for ingesting. <<<< - ING RWA is progressing well with the CD-to-disk dumping. There are about 2000 CDs left to process which should be completed by the end of January. MJI noted that the internet transfers were continuing to operate smoothly. - WFCAM MR said that the transfer of raw data to ESO is progressing, albeit slowly. Data up to 9 October has been transfered. The transfer speed remains unchanged even after ESO-end upgrades. The Cambridge Raw Data Archive has data ingested up to 12 December. He noted that a tape containing data for the period 13-22 December arrived on Wednesday 4th. - UKIRT There is no new data for the UKIRT archive at the moment. Probably unsurprising given the time WFCAM has been on. JRL said that he would ask if there is any more data that needs transferring and ingesting. <<<< 9. Optical processing update - VST MJI said that the VST UK-led public survey proposals ATLAS and VPHAS++ had been successful, but that we are unlikely to get any data to process this academic year. - IPHAS In lieu of VST data, data processing is carrying on for the IPHAS consortium for semester 05A/B (~80 nights of INT time). This should start to ramp down over the next year. 10. AstroGrid liaison update STH said that he had been unable to attend the AstroGrid deployment meeting at Leicester due a clash with the SV meeting in Edinburgh. 11. AOB. - GAIA MJI reminded us that the PPRP visit to Cambridge will be on the 9th February. This is an important part of our Gaia grant bid and potentially forms a significant part of CASU funding from 2007 onwards. A PSF and pipeline demo (or similar) will be required for this to emphasise the synergy between the different projects that CASU have been involved in. RGM suggested that some dry runs for this would be a good idea preferably before the end of January. - WFMOS MJI reported that he is involved in one of the two WFMOS concept design studies as part of the Aspen programme for Gemini 2nd generation instruments. He noted that there might be some future CASU involvement in this project (for background info see http://www.gemini.edu/files/docman/science/aspen/ WFMOS_feasibility_report_public.pdf It was also reported that STH has finished his term on the DAC, but that MJI has been co-opted onto the Staff Committee. PSB reported that there had been a site-wide power cut on New Year's Eve. One disc was lost on a CASU RAID array, but no data was lost. All machines are now up and running after a bit of hiatus. There were problems trying to reboot the machines of MR and EGS. It was requested that they sort them out so that they can be rebooted easily (and preferably automatically if they happen to be away) since they now serve a lot of the web-based information pages. <<<< JRL said that the ADASS 2007 venue had been sorted out and that the conference dinner (fish?) would be held at the National Maritime Museum. Continuing Actions ------------------ JRL finish off the OMP database access scripts JRL provide a script to add JAC MSB quality flag to the FITS headers MJI finish investigating merged classification statistics WJS advance the proposed CASU visit to RAL, in particular check that having it during AIT3a is OK New Actions ----------- EGS hold meeting to discuss the future design of VDFS web pages DWE MJI EGS generate stacked DXS data and catalogues by the end of January in liaison with Alastair Edge MJI produce colour-colour and/or colour-magnitude diagrams of IC4665 and M17 along with a brief howto MJI ask Konrad Kuijken about the ordering of OmegaCam extensions within a FITS MEF file MJI clarify deliverable date for DRL V1.0 PSB suggest a suitable strategy for jittering VISTA twilight flats JRL MJI DWE release latest PSF report JRL finish mods to AAT archive and inquire if there is any more AAT data that needs ingesting JRL ask if there is any more UKIRT data that needs ingesting EGS sort out an automatic and reliable reboot for apm14 and apm15 MR