24 October 2003 Present: MJI, JRL, PSB, DWE, STH Agenda ------ 1. Actions from last meeting 2. Comments on WFAU minutes 3. Report on AGOC visit 4. Report on ADASS 5. Comments on V1.0 WFCAM FITS header doc 6. Comments on SMB's Observation Software Detailed Design document 7. UKIDSS CSS and CFT positions and pipeline acceptance 8. AOB Minutes ------- 1. Actions from last meeting STH has investigated Sandy Legget's data. Although good quality, it is probably not enough to do extensive extinction investigations on since it only covers 3 hours of one night (0.5 in airmass). Sandy has pointed STH to an archive containing photometry files (reduced data) for every standard (also humidity etc. data). The investigation of this data will be done by the UKIDSS meeting (21 Nov). <<<< STH no progress has been made with the extension of the photometric standards to the south for VISTA, ongoing <<<< STH has almost finished sorting out document numbers for our released DWE documents. It was decided that we would follow VISTA's lead regarding version numbering, with the version number separate to the document number. Given the variety of methods (html, latex word) no formal template will be used, but each document will be headed by Title, Doc. No., Author(s), Date (centred). For now no logos will be added other than possibly the CASU one. STH has asked Richard Sword to slightly modify the current CASU logo since on b/w printers it currently reads "ASU" :-) Finish off numbering and email authors with the appropriate numbers. <<<< PSB the first draft version of the DID is done, and a copy was sent to MJI Steven Beard MJI has contacted Mark Casali regarding the QE variations of the 4 science-grade detectors, and is awaiting a reply, ongoing <<<< MJI needs to contact JPE regarding a list of the sections missing from the User Requirements Document and Calibration Plan document. <<<< MJI have sorted out their visit to the Astrowise group in Groningen. This JRL will be during the first week of December (3/4?). It was noted that members of WFAU would also be going. JRL has discussed the summit pipeline software plans with JAC staff at ADASS and resolved any outstanding issues. JRL has informed MJI of the things that need updating in the milestones document. The on-line version is now up to date. MJI has not contacted SJW regarding adding MSB design to the agenda at the forthcoming UKIDSS meeting because it is implicit in the memo that AL sent round (item 7.) and there is no draft agenda yet anyway. JRL is in the process of reducing the FIRES data. This will be useful for several purposes: supplying test stacking data for the Subaru group; demonstrating a different perspective for NIR data reduction cf. UFTI; and can also be used for "manual pipeline" comparisons. The results of this are need for the upcoming UKIDSS meeting. <<<< PSB met with JMS to discuss camera templates, nomenclature and other issues and had a constructive meeting. PSB met up with ESO folk at ADASS (see later). JRL MJI checked the on-sky characterization document - all ok for now 2. Comments on WFAU minutes It was noted that WFAU were investigating GridFTP. MJI noted that the summary of the AGOC meeting given in the WFAU minutes was very accurate. 3. Report on AGOC visit MJI reported that the pipeline progress, WSA updates and SSA demo (prototype UKIDSS archive interface) were well received at the AGOC meeting. The panel seemed to be satisfied that good progress was being made on these aspects of the programme and that both the CASU and WFAU parts of the project were generally on track. This appears to be the last meeting of this special advisory group and it is likely that future oversight will come from the full Grid Steering Committee (GSC). The next meeting of the GSC is likley to be in January where, among other things, an "enhanced" VEGA bid will be (re)considered. A meeting of the extended VEGA consortium will take place at the IoA on Tuesday 28th October. One clear message from this (and the previous) AGOC meeting was that a User Committee should be set up for VDFS (cf. UKIDSS). This should provide scientific and technical oversight and act as an interface to the user community. JPE is looking into this. Simple summary/overviews such as single page GANTT progress charts were requested by the committee. STH has produced a summary GANTT chart and updated the more detailed GANTT spreadsheet at the same time. These have been forwarded to JPE and placed (more prominently) on the web pages. The committee also suggested revisiting the issues of risk assessment and mitigation strategies in the light of WFCAM delays, and to look at prioritising VDFS deliverables for possible descoping in case of future funding or timeline constraints. 4. Report on ADASS PSB reported the following on his and JRL's ADASS trip: "Jim 'n' Pete met with Michele Peron, Benoit Pirenne and Adam Dobrzycki. We discussed pipeline infrastructure with Michele (as head of software development) and she reported that version 1.0 of the Common-Pipeline Library (CPL) was due to be issued in the New Year. We agreed that it would be beneficial to visit ESO and discuss CPL and the VLT pipeline infrastructure, at a date to be determined, but before Christmas. ESO would give us the release of CPL at that time. Benoit is concerned with archiving WFCAM data, and Adam is chairman of the Data Interface Control Board (DICB). It was agreed informally that o WFCAM should add the MJD-OBS keyword at source o CASU will, en passant, add a number of ESO-friendly archive keywords during assembly and transfer of the raw archive data to ESO. (For the record these will also be added as comment statements in the raw online MEFs) o ESO would provide a module, in C, to use networking protocol (HTTP probably) which would run in Cambridge to transfer WFCAM data to ESO via the internet o ESO will consider urgently and sympathetically accepting lossless FITS/TILE/Rice compressed data." PSB needs to arrange this ESO visit with Malcolm Stewart and Michele Peron. This would probably be 13/14 November. <<<< 5. Comments on V1.0 WFCAM FITS header doc Various comments had been sent in by email: a couple of typos (RA---ZN -> RA---ZPN and PV3_3 -> PV2_3) and a few misunderstandings about the structure of the WCS. MJI will collate these comments, add his own and forward them on. When this has been done the FITS header document can be signed off, as can the JAC CASU ICD. <<<< 6. Comments on SMB's Observation Software Detailed Design document It was noted that PSB will go to the Camera FDR during the week of December 15th. Various points were discussed. Since many of these were technical detail it was agreed to email them to MJI (by the end of this afternoon) rather than record them in the minutes. These points would then be assembled into a coherent whole, sent off to SMB and copied to the rest of the group, by the end of the day. <<<< 7. UKIDSS CSS and CFT positions and pipeline acceptance As a general comment, CASU agreed with the AGOC that there is an urgent need to set up a VDFS user committee to provide scientific and technical oversight and act as an interface to the user community. Both of these consortium positions should come within the remit of that committee. The CSS should be a member of the committee and the CFT should report to it. CSS task list: (1) It was unclear what the purpose of a review of these pipelines and archive procedures might be. The main comment was that such a review seemed far too late in the project. (2) The relevance of UFTI data to WFCAM and VISTA was questioned. The field of view is much smaller, the camera controller is different, and there are known problems with the stability of the results from the instrument. Practical strategies to deal with UFTI data are not necessarily optimal for dealing with WFCAM or VISTA data. (3)-(4) Fine (5) There was some general discussion on the desirability that MSBs should not have filter changes within them. It was also noted that the optimum MSB strategy to adopt for each survey depends on the (unknown) performance and stablity of the camera. (6)-(15) Fine CFT job description: It was thought that the job title "Consortium Food Taster" was not going to win any prizes and that an alternative title incorporating words like "data quality" or "science verification" would be more appropriate. (1) Fine (2) Some of the tasks eg. repeat data, seemed to overlap with what the Survey Manager is supposed to do. (3) Setting up the "sample science questions" seems like a task for the heads of the various survey groups. (4)-(9) Fine (10) CASU are not happy with the CFT authorizing data releases to ESO. This will introduce an extra layer of unnecessary bureaucracy into the system. And anyway how does (10) differ from (11), if the ESO community have the same access rights as the UK community ? However, this does raise the question of who will have authority over data releases ? (11) Regarding science verification the main question should be - are the pipeline and archive meeting the science requirements ? (12) Can this be partly equivalent to a user support role ? It was felt that the best use of this person would be to act in a support role for users and to help WFAU and CASU deal with queries from users. The CFT ought to be involved in the fine tuning of the system while the on-sky characterization and commissioning is being carried out (without necessarily having to be on Hawaii). The general consensus was that this sounded very much like a full-time post rather than 50% as described. A question that needs to be answered is what is the relationship between this post and the Survey Manager? Pipeline acceptance: There was then some discussion regarding the process of pipeline acceptance. This was also in response to a separate email from SJW. The overall comment was that it was difficult to see how this could be completed before commissioning and instrument characterisation since no battle-plan survives contact with the enemy. However, as part of a pipeline acceptance schedule, STH in discussions with SJW have suggested comparing our pipeline processing of (in order of preference) FIRES, UFTI and CIRSI data with "manual" processing. SJW has also suggested comparing EIS processing of ???? with the CASU pipeline. It was pointed out that JRL has already installed an UFTI version of the summit pipeline at JAC running under ORAC-DR and successfully compared the CASU pipeline output with the normal JAC ORAC-DR reduction. Further UFTI comparisons will be undertaken using the CASU pipeline in Cambridge to process the 6 sets of UFTI service data obtained by SJW. STH said that he would find the 6 sets of service data and pass it over to JRL. <<<< The FIRES data is in the process of being reduced - see earlier action. A lot of CIRSI data has already been processed. STH will ask SJW if he wants to pursue this aspect of the comparisons further. <<<< It was decided that there was no point in CASU processing further CIRSI data unless there was a compelleing science reason for so doing. Between them STH and JRL need to organise the FIRES and UFTI reductions by the November UKIDSS meeting. <<<< We all need to think about this issue further. How should the process of pipeline acceptance be carried out? <<<< 8. AOB MJI asked STH to look into the spatial systematics in UFTI data using Sandy Legget's photometry data. <<<< MJI noted that there have been a few top level alterations to the WFCAM/VISTA VDFS web pages to emphasise the VDFS aspect more. These pages can now reached from either the current /~wfcam or the new http://www.ast.cam.ac.uk/vdfs/ link. STH reminded MJI that the CASU entry web page has been updated and that there is a new description of the CASU activities for the IoA's graduate prospectus. PSB pointed out that with the upturn in VISTA activity we will have to be more precise in referring to activities in the minutes to avoid possible confusion. Continuing Actions ------------------ STH to finish investigating the supplied UFTI photometric standards files by the UKIDSS meeting (21 Nov) STH extend the number of fields that he has established for WFCAM to the south for use by VISTA. This is now a Q4 task. STH finish off numbering of documents and email authors with the appropriate DWE information MJI acquire details about the QE variations of the 4 science-grade detectors MJI contact JPE regarding a list of the sections missing from the User Requirements Document and Calibration Plan document. JRL finish reducing FIRES data in time for November 21st UKIDSS meeting New Actions ----------- PSB arrange ESO visit with Malcolm Stewart and Michele Peron ALL email comments concerning V1.0 FITS header document to MJI (by end of this afternoon) MJI collate these comments and send them off by end of today ALL email comments to MJI regarding Steven Beard's document (by the end of this afternoon) MJI collate these comments and send them off by end of today STH acquire (find?) the 6 sets of UFTI service data and pass it on to JRL JRL for reduction in time for the UKIDSS meeting STH to ask SJW if he wants to pursue the issue of CIRSI comparisons ALL think about how the process of pipeline acceptance should be organised STH look at the spatial systematics in the UFTI standards data