21st June 2005 Observatory meeting room 13.30 - 15:30 Present: MJI, RGM, PSB, DWE, EGS, STH, MR, JRL Apologies: WJS Agenda ------ 1. Actions from last meeting 2. Comments on WFAU minutes 3. Report on UKIRT Board meeting 4. VDMT report 5. Data archives update - UKIRT, WFCAM, AAO, ING 6. Optical processing update - VST surveys news 7. WFCAM telecon and processing update 8. VDFS update 9. AstroGrid liaison update 10. AOB - VISTA data disk space Minutes ------- 1. Actions from last meeting JRL has still to include access to "observing" log index files in the raw WFCAM archive - ongoing <<<< MJI are in the process of writing a new report on the detector properties JRL as they currently are - ongoing <<<< MR MJI has produced a colour picture of M51 and sent it to SJW. This may be used as a PPARC press release MJI placed the WFAU document on the global calibration of UKIDSS data on our internal web page EGS there were not many comments on the above document. MJI said that he would send in his comments by the end of the week and EGS will then pass them on <<<< ALL sent in our monthly reports. It was noted that the next VDMT will be on 8 July. This will be a face-to-face meeting held in Cambridge. STH requested that we send in our quarterly reports to him by 1st July. <<<< JRL has started work on the download side of the AAO archive scripts (see item 5) - ongoing <<<< WJS the latest version of the SADT Definition Document has not been acquired yet - ongoing <<<< MR renamed the Issue Collector in the Plone system to Reporting Database EGS PSB have discussed the responsibilities regarding cass123 STH RGM distributed the document by Tony Linde on service support for VO to the relevant people RGM have not yet had a meeting to discuss what equipment needs to be bought PSB from the AstroGrid money - see item 9 STH 2. Comments on WFAU minutes PSB said that the general impression was that problems were arising but being effectively dealt with, and progress seemed good. MJI noted that there was a bit of a backlog with the transfer of data to WFAU. Although the data is ready for transfer, it appears that WFAU have not copied it yet. Since 1st April, 6 nights have been transfered and 14 are waiting for transfer. This backlog amounts to about 1.5 Tb (Rice squished). 3. Report on UKIRT Board meeting MJI reported on the presentation he gave at the UKIRT Board meeting on 26th May. Although suffering from food poisoning, his presentation went down well, unlike the stilton from the previous day. The talk was similar to the presentation given at the WFCAM Technical Meeting/Brainstorming Session we held on 9th June and gave a general status report on the reductions. MJI asked the board why single 5s exposure frames were being taken even for the deeper surveys. He is not convinced that this is the best observing strategy. DWE was asked to look at the observing efficiency split by survey in time for the next JAC telecon on Friday evening. <<<< There then followed a discussion on how efficient are we in processing the data - can we keep up ? If the CPUs we have available are all used, then there is no CPU time problem. However, a related issue is that a careful eye must still be kept on the reductions and this can cause delays. MJI stressed that the persons on the rota dealing with the processing should have this task as their top priority and attempt to run the processing PCs as efficiently as possible. It was also decided that as a trial there should always be two people responsible for the reductions at any one time since the task was currently too much for a single person to easily manage. 4. VDMT report The last VDMT meeting took place on 3rd June. Preparation for the VDFS UK Review and the short timescale for this was discussed, but decisions on this were deferred to the next meeting VDMT. There was some discussion about the details of the URD and MJI provided some futher feedback on several items to WJS. 5. Data archives update - UKIRT, WFCAM, AAO, ING JRL reported that the UKIRT data archive was up and running and taking requests. JRL was reminded about removing the remnants of the old archive from the data disks on cass36. <<<< The WFCAM raw data archive is also up and running. A simple password system is in place. There will shortly be a link to this from the JAC web page and the CASU archive web page. Data from the 4th-12th June is being ingested, MEF'd and verified. Raw data up to 3rd June is in the archive. The last WFCAM data for this session was taken on 20th June. Currently we are 2-3 weeks behind the observing with the raw data archive and we are aiming at being 3-4 weeks behind the observing with the processed data. All the DVDs have been read and placed online for the AAO archive. 10% of these were however, unreadable. Replacement data is being obtained over the Internet. 60-70% of the user interface has been written by JRL. Reading the DVDs for the ING archive is in progress. However, there are problems with the DVD quality eg. in the latest batch 4 out of 5 DVDs were unreadable. It was noted that the processed WFCAM data will go onto our latest acquisitions: apm22_a and apm22_b. The data on these will be backed up on LTO tape. DWE said that he would update the hardware web page to reflect the latest changes. <<<< MJI commented that all of the WFCAM tapes received so far from UKIRT have been read successfully. This corresponds to 48 tapes and currently constitutes the CASU raw data archive backup, together with a copy of the (few) missing files. 6. Optical processing update - VST surveys news MJI is currently working on processing MegaCam data (with Nicolas Martin) and Subaru data. A new DLT tape drive has been bought to read the data tapes. EGS will be working on INT processing of some XMM survey data over the next few days. MJI will be attending VST Public Surveys meeting to be held in Garching this Wednesday and Thursday. This will be a round table discussion of the successful core and secondary programmes together with a discussion on data product deliverables. It is possible that a similar procedure will be adopted for VISTA public survey programmes. MJI will report back on this at the next CASU meeting. <<<< 7. WFCAM telecon and processing update The next WFCAM telecon will be held on Friday (24th June) at 9pm. Various issues need some discussion including clarifying calibration frame (eg. dome and twilight flats) data taking. <<<< MJI noted that Andy Adamson had sent an email suggesting that "crud" on the inside surface of the field lens could be causing the interesting features we see in the sky correction stages. This would explain why it appears to vary as a function of telescope attitude (external scattered light). A question arose about when WFCAM will next be on the telescope and taking science data. MJI will check the details of the planned schedule at the telecon. <<<< There was also some discussion about the date of the next UKIDSS meeting. RGM said that a recent email from Steve Warren had some information and he would distribute this. <<<< MJI said that the WFCAM processed DQC database is being developed and will be similar to the INT WFC one. Web pages monitoring the progress of the surveys, QC summaries and overall processing progress will be driven from this. MR will take over the development of this including scripts to update web pages showing the progress of the surveys driven from the WFCAM processed data database. 8. VDFS update PSB said that there have been a number of email exchanges regarding the VISTA FITS headers, the issue being the current absence of a set of VISTA-specific information, which is part of the agreed design, from the draft VTCS data dictionary. It is hoped that this issue will be resolved by the time of the release of the upcoming VISTA/VIRCAM simulator package. WJS had reported an equivalent verbal discussion having taken place. The conversion of the ETC from Perl to C has been done by PSB. (see http://www.ast.cam.ac.uk/~psb/vista/etc/) The date of the next VDUC/UKIDSS meeting needs addressing since we need some input from external users of the processed data in advance of completing the extra UK URD documentation. <<<< 9. AstroGrid liaison update Regarding the money from AstroGrid that could be spent on hardware, RGM said that he needs to get more information on this eg. how firm is the 1st July deadline and what is the amount ? <<<< RGM said that on 11 July the AstroGrid consortium will release V1.1 of the software. STH said that Kona Andrews has not been in touch with him recently so there have been no new requests. RGM wanted it noted that CASU had complied with all the requests so far from AstroGrid. There was then some discussion on what kit the money should be spent on. The consensus was along the lines of a dual processor PC with a 1-2 Tb RAID array. This should be within the envelope of the likely funding. RGM will send Tony Linde PSB's comments on the document covering the service support for VO. <<<< MJI said that he had fixed the cass36a disk in response to a request from GTR. This disk is serving Pleiades data to AstroGrid. 10. AOB - VISTA data disk space There was some discussion as to how much disc space the VISTA project would need in the near future and how soon. PSB said that the timescale was between now and autumn and the amount would probably be around 1 night's worth ie. 250 Gb. It was decided that this should go initially on apm22_b. Continuing Actions ------------------ JRL include access to "observing" log index files in raw WFCAM archive MJI finish off report on the detector properties as they currently are JRL and illustrate with processing examples MR MJI collate comments on the WSA Photometric Recalibration Design Document EGS and pass them on JRL finish off download side of AAO archive scripts WJS acquire the latest version of the SADT Definition Document New Actions ----------- ALL send in our quarterly reports to STH by 1st July DWE write an observing efficiency script (divided into surveys) by Friday evening JRL remove the remnants of the old UKIRT data archive from the data discs on cass36 DWE update the hardware web page to reflect the latest changes MJI report on VST meeting MJI raise flatfield (twilight and dome) issues at the next WFCAM telecon MJI ask about the WFCAM schedule at the next WFCAM telecon RGM distribute recent email from Steve Warren regarding next UKIDSS meeting MR take over the development of the WFCAM progress DQC database MJI discuss the date of the next VDUC/UKIDSS meeting at next VDMT RGM find out more detail about the AstroGrid hardware money RGM send PSB's comments on the document covering the service support for VO to Tony Linde