27th July 2004 Hoyle meeting room 10.30 - 12:30 Present: JRL, PSB, MJI, DWE, WJS, STH, JMI Apologies: RGM Agenda ------ 1. Actions from last meeting 2. Comments on WFAU minutes 3. VDMT quarterly meeting 4. Q3 work plan 5. Status of WFCAM - WFCAM ZPTs - JRL and MJI WFCAM crosstalk data from Morag Hastie 6. VDFS VISTA stuff - VISTA UK URD and science examples - dicHEADs - QFITS - RIX responses 7. Pipeline Processing update - COROT/IPHAS - virtual console server - APM2/APM3 rack mounting + RAIDs + virtual consoles 8. AOB - APM building news - Grant updates Minutes ------- 1. Actions from last meeting MJI One correction to the last minutes didn't make it to the online version. The correct version will be moved to the online location. <<<< MJI has made some progress on the comparison of the FIRES project reduced data with ours. The WCS parameters in the headers have been corrected and pipeline catalogues have been generated. MJI asked STH for the <<<< location of the FIRES catalogues. There was also a discussion about different seeing-weighted stacking options. Ongoing. <<<< JRL have written a document describing requested enhancements of QFITS PSB (+ some CPL). This should be circulated for all of us comment on. <<<< Further QFITS discussion was deferred until item 6. JRL has obtained a copy of the crosstalk data and Morag Hastie's report (see item 5). STH has emailed Simon Dye to request the generated MSBs that the survey heads are responsible for. JMI has investigated setting up an external (to the APM building) backup location. 2 GB of storage will be required. Where this would go was discussed. It was decided to wait until /data/superstore was set up (soon). rsync will not now be used since it requires the use of root privileges. WJS has checked the latest URD and found no major problems. ALL the latest URD (version 0.7) can be found at http://www.ast.cam.ac.uk/~wfcam/internal/vdfsdocs/vdfs-ukvurd_07.pdf Only a few comments in so far. The rest of us should read the new version and send in comments. Ongoing. <<<< ALL sent in monthly reports and quarterly data to STH. PSB have commented on RIX response document. WJS MJI JRL STH PSB has collated the RIX responses and sent them on to JPE. MJI has asked Peter Weeks regarding the storage of office equipment during the building work. There has been no reply so far since Peter is on holiday. MJI to chase up when Peter returns. <<<< 2. Comments on WFAU minutes There were no comments on the last two sets of WFAU minutes. 3. VDMT quarterly meeting MJI reported that STH had lead a fascinating and lively discussion on progress metrics. In a nutshell: Earned Value = Weighted Completion ie. the summation of the current estimated percentage completion of each task weighted by the prior estimate of the effort required for the task at the start of the project, normalised by the sum of the weights. E-Science funding was discussed. The Cambridge part of the VEGA bid for the period from October 2004 up to October 2007 involves ~4 FTEs. A JeS application (RG2 equivalent) for the Cambridge element of the VEGA award has been prepared and sent in by Paul Aslin. A statement of interest has been made regarding the processing of VST data which is now being juggled between assorted PPARC committees. There was some discussion of oversight committees and the possibility was raised of inviting external agents to attend the quarterly meetings. MJI said that JPE was keen on having a UK VDFS PDR, roughly scheduled for January. This would require a complete document set to be delivered by ~mid December taking into account the additional UK requirements. There was general concern that this may take a significant amount of work in an already busy period. It was noted that much of the documentation required for a UK VDFS PDR existed but would require significant reworking. The most efficient way of dealing with this would be to issue an extra set of documents in addition to those submitted (by then) for the ESO FDR. This all requires the timely completion and issue of the UK VDFS URD ie. before the end of August. The necessary effort to updated existing documentation and to draw up and document any new processing strategy will be included in the September Q4 work plan. It was noted that WFAU had requested CASU assistance in the selection of candidates for their 2 posts. This is anticipated to be at the shortlist stage. The next VDMT telecon meeting is a week on Friday (6 August). 4. Q3 work plan All were happy with the Q3 plan except PSB. He said that he would look at the plan and comment on it by the end of the day. <<<< 5. Status of WFCAM Steve Warren forwarded AA's WFCAM status update to all UKIDSS consortium members: all is OK except for the optics. The optics problems (FWHM 2-3x larger than anticipated) are caused by the corrector plate. Since the cost of a new blank is a significant part of the overall cost of the corrector, it has been decided to ship WFCAM immediately without the corrector so that the corrector can be refigured. Commissioning, mainly engineering, A/G and some detector characterisation, will go ahead as planned in September/October. UKIRT will then revert to normal use for the rest of the semester (November, December, January) in anticipation of the remaining commissioning taking place shortly thereafter. The upshot of this is that CASU personnel will not be required to go out for on sky characterisation until the New Year. A modest series of relevant commissioning tests can be done in the Autumn but this can be initiated and dealt with remotely. - WFCAM ZPTs The magnitude discrepancy in the zeropoints may have been resolved in favour of the more optimistic ones used in the CASU simulations and on the UKIDSS web pages. - WFCAM crosstalk data from Morag Hastie JRL reported on the crosstalk data received. Analysis of the data confirms what was in the report. The strange +/- profile of the crosstalk images could be due to a "bias" problem, however no bias frames were available to check this out. In order to get more information on this we apparently have to wait until September when the WFCAM PCs have been unpacked (sic!). Various low level patterns along (readout) rows were seen in the data which appeared in adjacent channels and even within other quadrants, but did not persist between frames. JRL was asked to acquire more data, specifically series of darks and flats from Derek Ives and to chase up the WFCAM acceptance test data from Paul Hirst. <<<< 6. VDFS VISTA stuff - VISTA UK URD and science examples MJI went through four science examples that had been forwarded by Tim Naylor to append to the current version of the URD. He also had a few more detailed comments regarding the 0.7 version which he will email to WJS. <<<< - dicHEADs PSB said that the data dictionary is supposed to be sent to Michelle Peron by 1 August. However, the VISTA-TCS dictionary has still not been issued, although the interim DIC contains a set of new VISTA-specific TEL keywords previously agreed. There was some discussion about how best to proceed and in the end it was unanimoulsy decided that he should proceed by forwarding the current (interim) DIC to JPE for him to send to Michelle. <<<< - QFITS PSB went through the main enhancements CASU would like to see in QFITS: * access to MEFs via [] notation, also extraction of arbitrary sub-image sections * 8-bit BYTE and 16-bit INTEGER data types * seemless RICE tile compression and decompression * a WCS tools library * ugraded error handling * a more comprehensive User Guide * CPL should handle more sophisticated weight maps (eg. confidence maps) - RIX responses PSB said that these have been sent to JPE for comment. It is unclear whether these have been passed on to Michelle Peron. STH will check on the status of this at the next VDMT. <<<< There was a general discussion of the need for a meeting (with Steven Beard, Malcolm Stewart, Jim Emerson) to sort out some of the wider-ranging issues arising from the RIX responses, and agree on the more straightforward ones. The earlier the but as August can be tricky it may be more pragmatic to aim for early September. MJI said he would raise this at the VDMT. <<<< 7. Pipeline Processing update - COROT/IPHAS JMI reported that he has used a 250 Gbyte ~1 week chunk of IPHAS data as a dry run of the processing pipeline and progress monitoring scripts. The r' and H_alpha data went straight through with no major problems. The i' data processing fell over due to the usual problem of not having an alternative master library fringe frame in the correct place. <<<< Splitting the data by filter to the different processing computers worked very well in this case since the persistence problem is negligible. One problem encountered was that access to USNOB via VIZIER (sic!) was way too slow (as expected) and that a local copy needs to be used if available. JRL will be incorporating a local version of the calibration catalogues to fix this problem. This is crucial for summit processing. <<<< [It was noted in passing that there was an online disc archive on La Palma of all ING data since Jul/Aug 2003. So it is now possible to get replacement files of corrupted or missing data eg. the attempted COROT data dry run test which was postponed due to tape reading problems.] The progress monitor worked fine during these trials, highlighting problem datasets as well as the overall pipeline progress. There is also a monitor which shows the status of the various computers. - virtual console server - APM2/APM3 rack mounting + RAIDs + virtual consoles JMI said that he had set up a virtual console server so that there was no need to have a console for each computer. From these virtual consoles you can access remotely the BIOS and perform various network operations which before had to be carried out on the computer's console, implying apm2 and apm3 can therefore now be rack mounted together with their associated RAID disk systems. <<<< 8. AOB - APM building news MJI said that various seemingly random people keep coming in to measure things, this may represent progress but ...... as there are no tenders for the work received yet, there are no firm dates. - Grant updates Paul Aslin is helping MJI sort out the various grant left-overs (remaining CASU budget, Data Archives support, VDFS operations bid, research etc.) in lieu of a September grant bid for such sundry things. Continuing Actions ------------------ MJI cp correct version of the previous minutes to the online location MJI compare FIRES project reduced data catalogues with ours ALL read QFITS wish-list and send in comments ALL read latest version of URD (0.7) http://www.ast.cam.ac.uk/~wfcam/internal/vdfsdocs/vdfs-ukvurd_07.pdf and send in comments MJI chase up Peter Weeks regarding the storage of office equipment during the building work New Actions ----------- STH provide location of FIRES catalogues to MJI PSB read Q3 plan and provide comments to STH by end of day JRL acquire any available darks or flats from Derek Ives and also acquire the acceptance test data from Paul Hirst MJI email his URD comments to WJS PSB forward DIC to JPE and suggest he in turn forwards it to Michelle Peron even though the DIC is still not complete STH to check up on status of responses to RIX response document at next VDMT meeting MJI raise question of organizing meeting to sort out some of the issues arising from the RIXs at next VDMT meeting JRL update pipeline to more gracefully deal with lack of fringe (or suitable flat) frames. JRL build in option of using local version of calibration catalogues in pipeline JMI rack mount apm2 and apm3 systems