6th July 2004 Obs meeting room 10.15 - 13:00 Present: JRL, PSB, MJI, DWE, WJS, STH, JMI Apologies: RGM Agenda ------ 1. Actions from last meeting 2. Comments on WFAU minutes 3. JAC/WFCAM news inc. Derek's visit 4. Report from SPIE meeting 5. MSB design issues 6. Status of WFCAM processing hardware + JRLs Sybase IQ kit 7. Status of UK VDFS URD 8. Results of CASU job interviews 9. Q3 work plans Q2 progress 10. AOB - RIX responses - APM building news update - next grant round bid Minutes ------- 1. Actions from last meeting STH has finished studying the spatial systematics in the UFTI standards data. Photometry is taken at 5 positions so that a sky flat can be generated. However, fringing or dust emission is potentially a problem. The data from each of these positions were stacked separately. No photometric systematics were detected at the ~1% level. This was carried out for the J and H data. With this limited spatial coverage (+/- 15 arcsec) no effect can be seen, however this is not an extensive test but it was felt that further tests should await WFCAM commissioning data. MJI no progress on comparison of FIRES project reduced data with ours (catalogues for the FIRES data are needed) - ongoing <<<< JRL reported that QFITS 5.0.0 has been released and progress has been made. Almost all items on our wish-list are there or can be outmanouvered. For example, a single extension from a multi-extension FITS file can now be extracted (although not by using the usual [] naming convention). The main thing missing is image compression. JRL was asked to draw up up a short (2 page) document summarizing our wish-list for CPL/QFITS. PSB was asked to help draft this and then circulate it first within CASU and then around the VDMT so that it could be sent to Michelle Peron by the end of July. <<<< ALL the job interviews for the CASU post have taken place (see item 8) MJI has sorted out the building name. The name agreed was the Astronomical Survey Unit - APM building. PSB has placed the VISTA Survey Planning document on our internal web site. It was noted that Martin Folger will be visiting ESO to discuss survey issues with them. JRL gave a presentation at the PPARC Data Curation Review, which appeared to be well received. The main unique observatory archives stored on the Cambridge server are AAO, UKIRT and ING. 2. Comments on WFAU minutes It was noted that PSB has also done some soak testing, of keyboards, and has found that Adnam's Broadside is particularly effective. There was general consensus with Nigel's viewpoint that we should carefully consider the cost-benefits of attending meetings ($700+ for a name badge at a recent meeting being a good example of part of the cost). However, JRL noted that an exception to this was the ADASS meeting in Pasadena (October 24-27). The deadline for abstracts is 2nd August. It is probable that JRL will be attending from CASU. MJI suggested making a start on the abstract and contents of talk using the SPIE abstract and submitted paper. 3. JAC/WFCAM news inc. Derek Ives's visit JRL reported on the meeting he had with Derek Ives. The main points of the meeting were to gain a better understanding of the read-out modes of the camera and the detailed orientation of the readout sector pattern. Currently all 4 science arrays are in the test cryostat and are being characterised. The cross-talk characteristics seem to only occur within a quadrant due to some nifty wiring. The persistence seems to have a very large time constant of about 1 minute (not an RC value) and appears to be independent of the number of resets. If this persists *groan* this may have some interesting consequences. Derek will do some flat field exposures for us to test the reset anomaly. JRL will obtain a copy of this data and of the report of the recent detector tests on a few week timescale. <<<< Due to corrector plate properties, the seeing will be much more stable and WFCAM will start out well-sampled with 2-3 pixel FWHM images. The current plan is to ship without the corrector, carry out a series of commissioning tests as scheduled and then eventually install a correctly- figured corrector plate. In a nutshell, the outcome of the acceptance tests were that the electronics and detectors have passed, but there were problems with the optics. The next telecon will be on the 14th July at which these and other issues will be discussed. 4. Report from SPIE meeting Nigel Hambly, Jim Emerson and MJI gave presentations at the SPIE meeting in Glasgow. No new issues were raised at the meeting. JPE pointed out in his presentation that ESO ought to consider the possibility of doing the first stage (ie. standard) pipeline processing, since they were carrying out most of the work for calibration purposes anyway (as we have been recommending all along). 5. MSB design issues All UKIDSS survey heads have now submitted plans. UDS will be using a similar plan to that of DXS. This appears to be a standard dither/ interleave pattern (fixed but order randomised) such that no image pixel resampling will be required (ie. small differential astrometric distortion). Offset skies (GPS) must be within an MSB in order to ensure that they are carried out contemporaneously with the main programme data. The LAS MSB selection will be dependant on the observing conditions. MSBs will range in duration from ~10 minutes to ~1 hour and will be interwoven with broadband (all) photometry MSBs roughly every hour. The minutes of the last UKIDSS Implementation Meeting (9/6/04) are available on the UKIDSS web pages. Since the last update for this was a few weeks ago STH volunteered to contact Simon Dye to find out what is happening. <<<< The current set of MSB designs will be processable with relatively simple stacking algorithms ie. no resampling. This will mean that the nightly MSB processing can be implemented using only the standard pipeline components. 6. Status of WFCAM processing hardware + JRLs Sybase IQ kit MJI reported that some of this was ordered about 2 weeks ago (after notification from PPARC that the extra money would be forthcoming) and has already arrived. The processing hardware consists of 5 rack-mountable twin Xeon processor 3GHz PCs each with a 3ware 1 Tbyte RAID array and 2 Gbytes of memory. They all have hot swappable everything including twin redundant power supplies (yes even these). JMI has installed Debian on these machines and they are all up and running in a single rack. This completes the standard processing hardware for WFCAM. Together with the earlier machine (similar spec.) we now have in place: an ingest server, 4 processing drones and a data products server. External mass storage devices are still needed and another backup LTO-II tape loader. Purchase of these items will be phased in over the next few months as required. The current plan is to test the machines by processing a 250 Gbyte chunk of recently received INT WFC data on them. For this test the data for each filter will be fed into a separate machine with the ingest server doing the bossing. WJS asked what would happen for WFCAM if all of one night was in a particular passband, could the processing cope since only one machine would be being used. There might also be problems with this scheme with eg. persistence. At this stage it is not 100% clear how linear the processing chain will be and what bits can be parallelized so this is more a test of machine inter-control and buck passing. It was noted the master processing software (/usr and /home) on the data ingest server is mirrored on the data product server (via rsync) every 24 hours, as a backup and spare. DWE/PSB suggested that it would be wise to also use a machine in another building to store a backup. JMI will investigate doing this on a 24 hour cycle <<<< JRL reported that he had been testing a version of Sybase IQ. This is a large read-only (ie. only update occasionally) database. The results have shown that it is very fast (PostgreSQL a transactional system is very slow in comparison). The remaining question is which version of RedHat Linux to use. According to the blurb IQ has to be run under RedHat (Enterprise expensive, Fedora hmmm.....), we don't know if we can run it under Debian. The database will be run on a dedicated fast PC with the same hardware spec. as the WFCAM processing machines. 7. Status of UK VDFS URD JPE has circulated a version of the UK VDFS User Requirements Document. This has supposedly only had restructuring carried out since the previous version, so it was suggested that it would be more ergonomic if WJS did a quick check of the document first for obvious typos before the rest of us comment on it. <<<< WJS said that some useful input has come from the VDUC members and a section on variable stars has been added. There was also some discussion regarding the Z filter. PSB said that even though the current finances do not stretch to buying a Z filter, the URD should task the pipeline with the ability to process such data. After WJS's checks we should all read the latest version of the URD. <<<< 8. Results of CASU job interviews The advertising for this post was deliberately low key both to test the water and to ensure we were not swamped with applications. However, as it happened, we received several promising applications (and several not so promising). Two candidates clearly stood out above the others, a post-doc from the IoA and a final year PhD student from ESO. Both were short-listed and interviewed and were unambiguously acceptable for the job. To cut a long story short, the ESO candidate was offered, and accepted, the advertised post, and will start working on VDFS-related issues from January next year. The IoA post-doc will work part time from October this year mainly on other CASU projects and potentially later on WFCAM processing/ operations. Longer term funding arrangments will be reviewed after the results of the next grant round bid. 9. Q3 work plans Q2 progress The next VDMT meeting with be on Thursday 15th July in Edinburgh. Progress reports for the last month, earned values data (days or %) for the last quarter and next quarter tasks to STH by Friday. <<<< 10. AOB - RIX responses It was decided that it would be more efficient for all those who attended the PDR (starting with PSB) to comment in turn on a hand-me-down copy of the recently received RIX response document. After it has done a complete round, PSB will collate and forward the collective wisdom to JPE. <<<< - APM building news update This is still on schedule (sic!). By 1st September the APM building has to be cleared of ALL junk. There was some discussion regarding where to move all the good junk to (the active computer equipment has an earmarked temporary home already). One suggestion was that we recolonize part of Greenwich House again. MJI said he and Peter Weeks (PW) from the workshop had already discussed this with Paul Aslin who saw possible interesting political ramifications. MJI and PW will do a bit more negotiating and report back at the next meeting. <<<< MJI pointed out that most of the newly acquired hardware would probably not be running while the building work was being carried out (because of the heat load in the office it was temporarily being relocated to). Probably only the ingest and data product servers would be powered up, but this should be enough to assess any WFCAM commissioning data and help fine tune the pipeline. - next grant round bid The next grant bid will need to be in by the end of September. This will cover the ramp down in the current CASU grant from 1st April 2005 and will include WFCAM operations, the rest of the Data Archive activities etc..., but not VDFS development or GAIA activities. - ad hoc MJI noted that Paul Hirst's current User Guide has WFCAM with a J-band sensitivity ~1 magnitude worse than those from previous estimates based on UFTI predictions and original WFCAM estimates. H-band has changed for the worse by ~0.6 magnitudes and K-band by ~0.3 magnitudes. MJI is currently investigating WTFIGO in conjunction with PH. Continuing Actions ------------------ MJI compare FIRES project reduced data catalogues with ours (generate catalogues for the FIRES data) New Actions ----------- PSB draft a short document summarizing our CPL/QFITS wish-list. Circulate JRL this so that it can be sent to Michelle Peron by the end of July. JRL obtain data and copy of the report of the recent tests from Derek Ives. STH contact Simon Dye to see what is happening regarding the MSB design. JMI investigate adding an additional backup to a machine outside the APM building. WJS check UK VDFS URD for faux pas and correct then send to ALL ALL then read and comment on the latest version of the UK VDFS URD ALL monthly reports and quarterly data to STH by Friday PSB comment on and circulate RIX response document WJS MJI JRL STH PSB collate RIX response document comments and forward to JPE MJI raise issue of storage requirements for detritus from APM building