24th October 2005 Hoyle Committee room 10.00 - 12:50 Present: DWE, MJI, PSB, STH, MR, JRL, EGS, RGM, WJS Apologies: NAW Agenda ------ 1. Actions from last meeting 2. Comments on WFAU minutes 3. Network transfers and UKLight 4. JAC telecon and WFCAM processing 5. VDFS VISTA update 6. Data archives update 7. Optical processing update 8. AstroGrid liaison update 9. AOB - APM building snags - GAIA PPRP bid - alternative RAIDs - CASU web pages Minutes ------- 1. Actions from last meeting MJI MJI has passed his comments on the SADT to WJS who is collating all WJS comments before sending them off - ongoing <<<< JRL distributed the location of the OMP database access scripts. He also checked which additional OMP tables were needed, the main one being the MSB status table. More work is being done on this - ongoing <<<< MJI has not finished giving feedback comments to MR on the flatfield and sky properties draft report - ongoing <<<< PSB contacted the University Computing Service to discuss the UKERNA pricing policy (see item 3) MJI has not provided URD feedback for the next VDMT meeting - ongoing <<<< RGM placed VDUC minutes on internal web pages and emailed their location JRL distributed the URL of the AAO archive interface, and has received comments back from the AAO and MJI MJI has started to investigate the merged classification statistics - ongoing <<<< MJI has not investigated the blended object complaints - ongoing <<<< MJI forwarded the details of the Edinburgh NIR meeting to JRL. Several of the group will be attending. There is a UKIDSS consortium meeting the day after also in Edinburgh. MJI said that he would contact Steve Warren to see if other CASU folk are needed at this, in addition to that well-known consort STH. <<<< WJS has contacted Kim Ward regarding a possible CASU visit to RAL. They have suggested either the first half of November or January. It was thought that a late afternoon visit during the week would be reasonable. WJS was asked to progress this action - ongoing <<<< MJI held a meeting with EGS, RWA and JRL to discuss ING archive issues JRL ensured that the current version of the pipeline does cross-talk correction JRL arranged the next JAC telecon (see item 4) MR set up a test harness using some of the data from 8th April. Some MJI additional data has been suggested, but has not been added to the test harness yet - ongoing <<<< JRL have discussed recent bugs in the V2 reprocessed data MJI MR investigated the web-based presentation of various changes to software A first draft can be found at: http://apm15.ast.cam.ac.uk/wfcam/2005a_version_notes/ There will be a version log for each semester, e.g. version 1 processed data for 2005A is not the same as version 1 for 2005B. We were encouraged to check this and provide comments. <<<< EGS had a meeting to discuss the group's web pages (see item 9) DWE MJI forwarded EGS's email regarding web pages to DWE MJI forwarded the email from Peter Weeks regarding the snag list to all. ALL comments and items were added to the snag list (see item 9) 2. Comments on WFAU minutes PSB noted that the minutes (and attendance) were shorter while Nigel was away. The only other comment was from RGM who noticed that a release date for Version 2 of the reduced data had not been mentioned in the minutes. 3. Network transfers and UKLight PSB said that he had found out that the body to lobby regarding the charging for our Internet usage is the IT Syndicate. The current charging model is that the university pays an agreed fixed cost for the year which is then split by usage between the various colleges and departments. The change to charging according to usage was relatively recent and lead to a short discussion regarding the costs of data transfer. The Computer Service is investigating the UKLight option. PSB was asked to get the latest news on this. <<<< PSB and MJI were volunteered to contact Paul Aslin regarding getting the IoA as a department to lobby the IT Syndicate with regard to changing the charging model. <<<< 4. JAC telecon and WFCAM processing The last JAC telecon was on 19 October. MJI, JRL, STH, Paul Hirst and Andy Adamson attended. MJI reported that main topics discussed were the status of data processing, pipeline versions, timeframes for reprocessing and data release to the community. The basis for much of this discussion was a report that MJI had drafted for including in the JAC report to the UKIRT Board Meeting on 17/18 November. The issue of PI's accessing their processed data was raised and MJI noted that CASU were occasionally supplying PI's directly with images and catalogues as a temporary measure. Non-standard data taking was also discussed and it was agreed that JAC would check with CASU on the viability of pipeline processing this type of data in advance of data being taken. The issue of the older commissioning data from November 2004 was raised. MJI commented that there were no plans to reprocess this with the current pipeline version unless there was a compelling reason. JAC will inform CASU of any further requirements for this later. JAC were updated on the various CASU web pages monitoring the WFCAM data flow and survey progress. Various aspects of flagging failed MSBs were also discussed. Ideally this information should go all the way through to the archive. The question is how ? since the information resides in an OMP database table and not in the FITS headers. MJI said that he would be discussing this with WFAU. After the field lens had been cleaned the throughput had improved by ~2% and no significant sky crud marks were now present. Also reported was an intermittent 60Hz pickup - apparently only easily visible in darks. JAC reported that it is now common to get seeing of 0.6 arcsec. MJI confirmed that the recent 05B DQC plots showed that 0.6 arcsec was being achieved on good nights on all detectors. DWE gave a short report on the astrometric properties of 2MASS derived by comparisons with CMC14 and UCAC. On average, the complete (rms + systematic) external accuracy per point source was 80 mas with this worsening to 120 mas at the faint end. A tentative offset of 20 mas with respect to the ICRS was found in declination. A more detailed estimate of the error as a function of sky position showed that the bright end error varied between 50 and 100 mas depending mainly on Galactic latitude. No large magnitude terms were observed. The bottom line is that 2MASS is well within specification for use as the astrometric calibration for WFCAM. MR showed the latest incarnation of the DQC archive. Various design aspects and perceived problems were discussed. It was decided that an offline meeting was needed. MR said that he would organise this. <<<< EGS showed his latest design for the survey progress plots. STH suggested that this go on the main VDMT web page. Various suggestions regarding the design were given. STH said that he will update the Photometric Calibration document either today or tomorrow. Not much has changed since his last verbal report. He noted that there is a hint of a spatial systematics at the +/-1% level. MJI asked for an update from DWE on the progress of the PSF fitting on the 05A data for the next meeting. <<<< 5. VDFS VISTA update WJS said that the camera is currently pumped out and cold. The cooling system is good enough that it can almost maintain this using only two out of the three pumps. PSB reported that Steven Beard had issued a major release of the camera software which now includes full WCS support. MJI asked about the transmission curves for the filters currently in hand. WJS said that these were available, but that for some of them a hot and cold measurement may not have been carried out. MJI noted that improved estimates of throughput would be useful for updating the ETC web form and plots. <<<< JRL said that he had been working on 4 CPL recipes and reviewing what is needed with them. The next major update is to add full error handling. 6. Data archives update UKIRT - Nothing new WFCAM - MR reported that all the tapes received so far (up to 10 October) had been dumped to disc. The data up to 21 September had been MEF'd and the main processing had been started. The transfer to ESO of raw UKIDSS data will be started soon. It was noted that apm23_b appeared to have a disc controller problem on Friday. No data had been lost though this was interrupting the MEF'ing. AAT - JRL said that he has had some feedback from AAO regarding the AAT archive interface. He is currently working through these suggestions. ING - MJI said that we had recently been sent 2 "naked" internal 400 Gbyte IDE disks as a transfer test to fill in the date gap between received DVDs and the internet transfers. Neither was fully readable. Further tests (eg. returned to sender) are being pursued. 7. Optical processing update EGS said that he had started processing the IPHAS June data. The processing of the COROT data had been finished. MJI noted that three UK-lead VST public survey proposals were submitted on time. CASU have agreed to process data for all three, proposals, which if successful will mean that we would be processing ~20% of the VST data. 8. AstroGrid liaison update EGS said that the recent AstroGrid Oversight Committee meeting went well and the demos were well received. 9. AOB APM building snags - the recent problems with doorlock demise and keys were discussed and it was agreed to keep the current set up as it is more amenable to general IoA use, providing that the last CASU person out uses the Chubb deadlock to double lock the building at night and weekends. The main additions to the snag list were heating and noise. The former has now been fixed and Peter Weeks knows how to reset things if MJI is not around. The latter is trickier but is under investigation. GAIA PPRP bid - MJI reported that this involves several CASU staff and was submitted last week. He proposed a vote of thanks to Nic, Floor and Francesca for the huge amount of work they had put into this. alternative RAIDs - PSB suggested an alternative to the current storage hardware that we have been buying. Recently an external fibre channel RAID array of 6.8 Tbytes (single partition) has been installed on a machine on the Solaris cluster. SCSI versions of these are also available. Although similar in price/Tbyte an advantage of these is the (in theory) smaller amount of support required since we can add them as external devices to our existing PC data storage bricks. MJI and PSB are investigating further. CASU web pages - DWE reported that he and EGS had a meeting to review these. It was agreed that one of the problems was that it was difficult to find some things. DWE said that a mini-Google search box could be placed on the HTML pages to assist people. It was also felt that having a consistent look and feel was quite important in order to have a professional look. The two strawman options proposed are either a fully Plone system or a simple HTML (using cascading style sheets) with an occasional tool running from Plone. A pro/con list will be drawn up. A list of options will be distributed in the near future in order to canvas all opinions. CASS36 - PSB asked if upgrading cass36 to Solaris 10 would cause any problems. The expected downtime would be about half a day. MJI said that we could survive for that long. CASS38 - PSB asked about this one also, to several *groans*. One problem, among many noted by PSB and JRL, is that the Sybase software on it will also needs updating. This one could run and run and needs scheduling carefully. VDMT - STH said that the next VDMT meeting will be on 8 November. All please send in their monthly report to him by the end of Thursday 3 November. <<<< Hardware - Concern was expressed that the servers for the Plone system and various databases were also the main processing machines for EGS and MR. A dedicated machine(s) is planned but it was decided that this issue should be revisited when the web site issue has been resolved. MJI asked for reassurance, and was given it, that the whole Plone system and associated databases were backed up on alternative PCs. Continuing Actions ------------------ WJS pass on SADT comments to Martin Folger JRL finish off the OMP database access scripts MJI feedback comments to MR on flatfield and sky properties draft report MJI append more details to the URD feedback for the next VDMT MJI finish investigating merged classification statistics MJI investigate blended object complaints WJS finish sorting out the CASU visit to RAL MR finish the test dataset harness MJI New Actions ----------- MJI contact Steve Warren regarding agenda for UKIDSS consortium meeting ALL provide comments on web-based presentation of software changes to MR PSB inquire of the Computer Service regarding UKLight MJI contact Paul Aslin about possible lobbying of the IT Syndicate about PSB the Internet charging model MJI arrange meeting to discuss design of DQC archive DWE update for next meeting on progress of PSF fitting of 05A data WJS provide updated measurements of the VISTA filter characteristics ALL send in monthly reports to STH by end 3 November EGS produce set of options for group web pages and circulate DWE