19th September 2006 Hoyle Committee Room 10:00 - 12:40 Present: DWE, MJI, JRL, PSB, STH, EGS, RGM, WJS Apologies: MR Agenda ------ Draft Agenda: 1. Actions from last meeting 2. Comments on WFAU minutes 3. Report from VDMT meeting 4. Report from Dubrovnik conference 5. Data archives update 6. Optical processing update 7. WFCAM news 8. VISTA news 9. UK review 10. VISTA public surveys 11. AOB Minutes ------- 1. Actions from last meeting JRL has not implemented the fix to the problem with the UKIRT archive (missing header information problem) - ongoing <<<< JRL mentioned that there has been some difficulty in reading the tapes required. DWE reminded ALL about the location of Plone site. A web new server has bought and installed. STH has been in contact with Albert Zijlstra to acquire the tiled supernova remnant image. The image will be forthcoming - ongoing <<<< STH has not investigated the effects of stacking on the 2MASS/WFCAM non-linearity comparison - ongoing <<<< EGS decided on a specification for the CASU web server, ordered it and PSB got it working. This will also be the location of a number of our MJI services in future. There followed some discussion regarding which flavour of Linux was being used (Debian, Ubuntu, ...) DWE has added the casuhelp email address to the bottom of each Plone page STH has contacted SJW to place this email address on the UKIDSS website and advertise it. This has not happened so STH will prod him again. DWE noted that the WSA site refers to the new email address (thanks). JRL has contacted AAT regarding the PostgreSQL problems so that he can repair the AAT archive. He is awaiting a response - ongoing <<<< EGS has made the new ING archive interface live, but it was noted that this was not the default one. EGS was asked to make the new interface the default one and provide a link to the old one - ongoing <<<< JRL has emailed AJA about fixes to the UKIRT archive MJI have not emailed the new WFCAM colour equations to DWE and JRL since STH they are still tweaking the Y & Z calibrations - ongoing <<<< PSB have discussed a proposal for RECIPE keywords, but have not come to JRL an agreement yet - ongoing <<<< MJI It was also decided to bring in WJS in on this discussion. STH has not provided JRL with new figures for DRLD PSB emailed JPE about an ICD for the Garching-Cambridge interface. JPE said that he would raise the issue with Martino Romaniello. 2. Comments on WFAU minutes RGM noted that although the data for some parts of 06A has been copied to WFAU, the flat file access to these files is not yet available. RGM pointed out that flat files should be made available almost immediately after the data has been copied. The use of cheap IBM machines was discussed. This was probably not considered a good option for CASU. It was pointed out that these machines might have a higher than average infrastructure requirement. PSB said that there had not been any progress on the UKLight front since Edinburgh computing services have found the diplexers. PSB said that he would send an email to see if the situation could be advanced. <<<< 3. Report from VDMT meeting STH reported on a UKIDSS Survey Heads telecon held on 7 September 2006. The minutes can be found at: http://wiki.astrogrid.org/pub/UKIDSS/VarDocs/telcon-minutes-20060908.pdf One of the main topics discussed was the DR2 data release. It is likely that this would not be until February 2006 due to the overheads of reingesting. The ingest of 06A will take place in December and the whole process will take a few months. DR2 will consist of 05A (V3), 05B and 06A. UDS is not part of the reprocessing, but the stacks will be available. It was mentioned that UDS have requested some further nights to be reprocessed but neither MJI or MR know which ones yet. MJI said he would email and find out. <<<< The new calibration was discussed and the next release will be with the new (final?) calibration. There was some discussion about quality flags. In particular, where should the cross talk flag come from, the catalogue or archive-end? MJI is still of the opinion that it should be science demand driven and therefore considered that the latter would be the more sensible option. Checking of the reprocessed (V3) 05A data was raised, however where this has got to is not clear (the flat file access was supposed to speed this up). There will probably be a UKIDSS future meeting to discuss this. The telecon also covered issues of scheduling and planning for 06B. STH added that there are discussions ongoing with the University of Hawaii regarding adding a part of their observing pot into UKIDSS. MJI reported on the VDMT telecon held on 4th September. Most of the actions on CASU result from the forecoming UK design review. It is likely that a PPARC representative will attend the review. 4. Report from Dubrovnik conference DWE reported on a very enjoyable image processing workshop in Dubrovnik. The main thing gained from this was assurance that our effort toward PSF fitting is worthwhile and that there are no problems with the way that the PSF software is being developed. It confirmed the original assessment that existing PSF software was not automatic enough and that bespoke software was the right choice. It also confrimed the CASU view that the only potential gain from PSF fitting is in very crowded fields. DWE noted that the general view among pundits present, toward the background estimation problem, was now veering away from localised fitting toward the global approach that we have pioneered and have been using for the last 25 years. DWE also noted that a significant number of particle physicists attended and are likely to be working on LSST in future. They have a lot of experience working with very large datasets and in detecting needles in haystacks. In fact they have done such a good job that they are no longer required. There was some discussion on uberCalibrations ie. using overlaps as additional information in your absolute calibrations (photometry and astrometry). There have been problems with the uberAstrometry and DWE pointed out CASU's use of unterAstrometry since there are enough standards to calibrate (wide) fields directly. DWE tried to get the topic of persistence discussed, but all that was said was that it was not removable from the data and that the best that could be done was to flag affected images if required ie. the same conclusions we've arrived at already. 5. Data archives update MJI said that all was quiet on the archive front, no problems and no news, other than as discussed earlier. 6. Optical processing update The main optical processing that is currently being carried out was caused by recovery from the failure of /data/cass55_c. As reported before, this 7 Tbyte RAID system has now been absorbed into the linux emporium, thoroughly tested and no problems found - even after our glorious power cut w/e. 7. WFCAM news RGM said that the first redshift 6, =nint(z), QSO from WFCAM has been detected and confirmed on the NTT. In carrying out this search crosstalk contaminants have been successfully identified and removed. RGM thought it ironic that the latest crosstalk removal algorithm makes it harder to identify very faint crosstalk contaminants. MJI asked if classification information had been used in the weeding since residual cross-talk artefacts tend to be weird shapes. RGM said he would find out and report back. <<<< All the WFCAM processing has been completed. The checking of this data is progressing slowly. MJI asked JRL to add the detection filter FWHM as a command line option in the pipeline version of imcore. <<<< DWE said that he had a long discussion with Phil Lucas regarding PSF fitting in the GPS fields. Phil is keen to get involved in part of the testing process. It is important that the WFCAM PSF routines get tested with respect to the more manual PSF software that exists. This will provide a valuable benchmark. JRL said that we are still waiting to hear from JAC if there is a solution to the MSB id checksum issue. It was noted that the 1st month of V3 05A has been ingested. As usual a few files had interesting foibles present. These have all been fixed. 8. VISTA news WJS reported on the latest news from the telescope. The Paranal site is running on its own generator and it is still likely that it will be doing when the camera gets there. The telescope acceptance is scheduled for the second week in October and currently all looks OK. He reported progress with the primary mirror polishing and noted that the secondary mirror appears to meet the wavefront specification. A finder telescope will be installed in November/December for telescope and dome tests. Also a 25cm baby mirror is still being considered so that some tests may take place even though the primary is not installed (the lead time for this is 3-4 months hence a decision will be made pronto). PSB said that he visited RAL last Tuesday (12th) and took part in acquiring further VIRCAM test data. This will be ingested and digested in due course. There will definitely be no more data available until the camera is at Paranal. The camera is now being warmed up. JRL's initial reaction was that the new replacement chip is much better than the old one. It was noted that a minor upgrade has been carried out to the ETC regarding pawprints. PSB said that he still has to update the documentation about this. <<<< There was some discussion regarding the concept of a half-filled tile. UltraV (the project that merged all deep field proposals) may want this. It was thought that the observing template could probably cope with this. 9. UK review The document list for the UK review was discussed. JRL said that he would email Sandra Castro to find out what the hardware requirement for the summit will be. <<<< MJI said that he had recently been working heavily on the technical paper. Since there had been no comments forthcoming regarding the Plone site a volunteer was asked to carry out the testing. JRL volunteered. <<<< MJI gave further details of the review. Planned Start 24 October Tuesday 10:30 End 25 October Wednesday 15:30 Lunch will be provided on both days and it is expected that about 20 people will be involved in total not necessarily all at the same time. There will also be an evening meal organised for those interested on the Tuesday. The documents will be send out on 30 September and give about 2 weeks for the panel to ask questions. It is intended that the answers will be discussed at the review. There will be 3 brief presentations given before lunch on the first day, followed by questions and answers. MJI reminded us that the review panel will be chaired by Luc Simard and consist of Gavin Dalton, Tim Naylor, Matt Jarvis and Bob Nichol. 10. VISTA public surveys VVV (bulge mainly time series + bit of Galactic plane survey) VIDEO (deepish) UltraV ('kin deep) Viking (deepish) VMC (Magellanic Clouds) VHS (shallow) The above are the current set of VISTA public surveys to go into the playoffs. This set of proposed surveys illustrate the range of different expectations of VDFS users ranging from: crowded fields, shallow wide-area surveys, very deep surveys involving iterative stacking, time-series analysis etc. It may be useful as part of the oral presentation at the UK review, to highlight these aspirations and address what expectations are met by the UK requirements and what are not. 11. AOB Power cut went as well as could be expected apart from wiping out late Friday afternoon, the entire w/e and a good chunk of Monday morning recovering from it. There were no serious problems with any of the machines in the APM building. PSB said that site-wide the only failure was one disk and one CUDN network switch (who needs the outside world). APM3: We were reminded that the home disks of apm3 have now been united with those of apm6-24. Quarterly reports: STH reminded us that quarterly reports will be needed by the end of the month. <<<< It was noted that a Gaia CU5 meeting will be held in Cambridge on 25/26 September. STH said that FvL was willing to have a meeting with the group in order to see where people might see themselves fitting into the CU5 plans. ADASS: JRL reported on progress with the catering for the ADASS 2007 meeting. Judging from an arduous taste test carried out by JRL and PSB, the choice made was very good on quality and quite reasonable on price. These caterers will be responsible for the reception and the breaks (tea & coffee). They will probably also set up a "cash lunch" area to provide for those who do not wish to forage around Kensington. The conference poster was shown and liked by all (apart from minor dissent by MJI regarding the artificial stars). The conference dinner is becoming more problematic and is still being investigated. Continuing Actions ------------------ JRL finish the fix to problem with UKIRT archive (missing header information) STH acquire tiled supernova remnant image STH investigate effects of stacking on 2MASS/WFCAM non-linearity comparison JRL repair AAT archive mirroring (PostgreSQL problems) EGS make the new ING archive interface the default one MJI email new WFCAM colour equations to DWE and JRL STH PSB agree on a proposal for RECIPE keywords and discuss with SMB JRL MJI WJS New Actions ----------- PSB email WFAU to see what is happening regarding UKlight MJI email Omar to find out which extra UDS 05B nights need reprocessing RGM report back on classification apsect of cross-talk weeding JRL add detection filter FWHM choice as command line option of pipeline version of imcore. PSB update ETC documentation regarding pawprints JRL email Sandra Castro to find out about the Paranal hardware requirements JRL test out the Plone site as if external mutt ALL send quarterly reports to STH by end of month