11th December 2007 Hoyle Committee Room 10:00-12:45 Present: STH, EGS, PSB, MR, MJI, DWE, JRL, RGM Apologies: RWA, NAW Agenda ------ 1. Actions from last meeting 2. Comments on WFAU minutes 3. Recent meetings/visits 4. Data archives update - AAT, ING, WFCAM, UKIRT 5. Optical processing update- IPHAS IDR 6. WFCAM news - raw data transfer status - WFCAM network transfers status - processing status - reprocessing status - photometric calibration status - UKIDSS workshop - DR2,DR3 checks 7. VISTA news - pipeline progress - camera test data - VISTA QC and operations - ESO SMP letters to PIs 8. ADASS 2007 9. AOB - grant review news - computer systems - updated plone site - APM christmas party Minutes ------- 1. Actions from last meeting MJI forwarded the details of the JAC wiki page containing the WFCAM observing summaries. MR has not advertized the illumination correction information - ongoing <<<< STH said that he would check these by applying them to DR3 and would put in a description of how to use them in the calibration paper. <<<< PSB acquired a wireless router for the APM building using the new order book which he also obtained. MR has not yet performed the backup verification tests using 4 old LTO tapes - ongoing <<<< MJI made the latest reports to the UKIRT board available on the internal pages. RWA organised disposal of the old data tapes. MR emailed ESO regarding the raw data transfer interruptions. The solution might involve the reinstallation of the NGAS server. It was noted that informal discussions next week may be useful. ALL did not provide comments on the photometric calibration paper since they had misread the minutes. Anyway, the advertized location was an old version - ongoing <<<< [STH emailed the latest version to everyone during the meeting.] MJI raised the issue of science verification of VISTA data at the next VDMT meeting. CASU collectively will attempt to find out more about this at ESO next week. EGS emailed JMI regarding the details of the database on apm1 and has now moved it to apm26. The AstroGrid registery has been updated. DWE will switch off the server on apm1, wait until after Christmas and (if there are no complaints) remove the files. <<<< MJI has not reviewed the user accounts on the APM cluster and will do so over the Christmas period - ongoing <<<< 2. Comments on WFAU minutes Congratulations on getting DR3 out of the door. We also wish JB well as he goes down the hill to move up to a new position at Geophysics and send the advice that, as he studies how the earth moves, be careful not to get stuck between a rock and a hard place. CASU also noted the resignation of JMS from the ATC and were curious if this included the VDMT as well. 3. Recent meetings/visits MJI noted that there had been a VDMT meeting on the 8th November and recollected the news that the VISTA NB and Z filters had been shipped and that the transmission curves had been measured at the company. PSB wondered if they were available. STH requested that we all send in reports covering November and December to him by 3rd January. The next VDMT meeting will be 7th January. <<<< 4. Data archives update Plates: RWA reported that the old ING data tapes are being cleared and that he is making good progress in sorting out the extensive and eclectic plate collection. He and REMG, who was visiting for 3 days, divided up the collection into various categories - those plates to go to Laundry Farm, those to be returned to originators, and a number to be given to the Normal Lockyer Observatory in Sidmouth who will collect soon. RWA has already mailed off a few small collections to their owners. The Indian Institute of Astrophysics at Kodaikanal want their 12,000 solar plates back. Rutherford will take the Cambridge solar plates (another 11,000) + disk prints (5,000). The remaining problems are the survey plates and films. December "Observatory" advertizes the POSS plates - the IIA may be interested in some of them. If neither works out we'll think again. RWA also noted that the room at Laundry Farm is being reconfigured to create space for the Cambridge plate archive. AAT: JRL noted that the AAT archive is about 2 months behind due to lack of transfers. He will contact Chris Rammage to investigate the hold up. <<<< ING: transfers continue seemlessly. Data to the end of November 2007 has been checked and ingested, and December transfers are up to date. WFCAM: MR reported that most of the 07B data, bar the last week, has now been received and ingested and that transfers to ESO are up to date. The last night of 07B WFCAM data was the 3rd December. It goes back on for 08A on 4th February. UKIRT: no change 5. Optical processing update EGS reported that the IPHAS IDR based on Astrogrid software tools has taken place and has been well received. The release covers about 50% of the Northern Galactic Plane and contains r,i,Ha data for some 200 million stars. A paper describing this was posted on astro-ph and submitted to MNRAS and a press release was sent out. This generated a lot of media interest helped by the colourful illustrative images. The rest of the group congratulated Eduardo for the success of the IDR and thanked him for the large amount of effort he has put into this. 6. WFCAM news Transfers: MR noted that data for a couple of nights with apparent tape problems were transferred from the JAC FTP server directly. As noted previously, we are one set of tapes shy of acquiring all of the WFCAM 07B data. PSB noted that no new information about the summit-to-sea-level gigabit link as been received, nor about the status of the proposed summit data server. MJI noted that we can follow this up next week with Luca Rizzi and AJA during the ESO UKIDSS meeting. Processing: Standard pipeline processing is on track with virtually all of October, bar one night still running, finished. All the September data has been fixed, checked and flagged as ready to copy. Reprocessing: The parquet floor problem persists through the October data and as noted previously has required us to adopt the following procedure: a. for each night we run the pipeline as normal b. check the skies for banding - which seems a reliable diagnostic c. if banding is present it seems to affect most (CDS and NDR data) frames in a night. d. banding in UDS, DXS, GCS, GPS products has so far been negligible due to stacking a lot of frames or crowding hiding any sign of it e. all Z, Y, 1-0S, nbJ and LAS J individual science frame runs from the pipeline output for nights with banding present are then corrected for any channel banding for all detectors (since it potentially can, and has, occured on all detectors at various times) f. we then rerun the pipeline for these files from the interleaving/ stacking and catalogue generation phase. This and the monthly detector zero-point updates are delaying normal release of data by about an extra 2-3 weeks. So far we have had no complaints about this extra delay but will continue to monitor the situation. Photometric calibration status: STH reported on the recent progress made on assorted calibration issues and as noted earlier emailed the updated draft of the calibration paper to the rest of the group for later comment. UKIDSS workshop: There was a brief discussion on who is presenting what next week at the ESO UKIDSS workshop, viz JRL: pipeline overview and 2D image processing; MJI catalogue properties; STH photometric calibration. DR2,DR3 checks: RGM reported on work on stellar classification based on a comparison of SDSS spectroscopic classifications with the CASU LAS UKIDSS catalogue classification. This was motivated by the need to determine the completeness of a sample of stellar extremely red objects that are only detected on H and K. UKIDSS LAS DR2 was matched with the SDSS spectroscopic database. There were a total of around 19000 UKIDSS objects with SDSS spectra, of which around 16000 are classified as galaxies and around 3000 are classified as stellar i.e. stars or quasars. 1650 UKIDSS LAS DR2 sources are classified as stellar on BOTH the SDSS images and also have SDSS stellar spectral classes i.e. quasars are excluded. The discrete CASU classification categories in the K-band are: -2 (probable star); 9 -1 (star-like); 1362 0 (noise); 10 1 (non-stellar); 269 Thus around 20% of SDSS stars are classified as non-stellar on K images. If the band-merged class is used the fraction drops to around 7%. These values imply that samples of stellar objects selected from WFCAM data in this way will be incomplete at this level. Most of the non-stellar classified objects have a classification statistic (ie. stellarness index) in the range 1 < kclassstat < 3. kclasstat > 3 objects are caused by blended sources in the UKIDSS data. The separate issue of the reliability of the classification is being worked on and a report which will use UKIDSS DR3 is in preparation. RGM also reported on a CASU catalogue deblending feature where some quite well resolved bright objects near other bright objects are absent from the CASU catalogues. This was a feature/effect that was present in earlier releases. He has provided a gallery of 11 examples from 2005, 2006 and 2007 data to MJI and JRL. Absolute numbers for this effect have not been determined since they came out of a search for extreme objects. Perfection may be impossible but he thought it would be worth adding this effect to the list of current known features. MJI commented that he had looked at a recent example from this set (20070421) using the toolkit development version of the software and not found any problem. He also re-ran the current pipeline catalogue software to check if this is still an issue there, and found that it was. He asked JRL to investigate why there appears to be a difference between what should be the same algorithm. <<<< Future of UKIRT: JRL volunteered to draft a response letter of support to Omar Almaini and Phil Lucas regarding the future of UKIRT/UKIDSS since not completing the UKIDSS surveys would be a rather strange decision. <<<< 7. VISTA news VISTA: WJS reports that the primary mirror now has a good fringe pattern right out to the edges, meaning that the extent of the remaining figuring is understood. MJI noted that the recent Chilean earthquake had not impacted VISTA or the VLTs. It is expected that the new Z filter will be delivered in time for first light. PSB mentioned we are still waiting for the N118 transmission curve as well as the new Z data, for the ETC. Software: JRL reported that work continues with the DRL, in particular the linearity analysis is being investigated using real VISTA linearity sequences, with extended study of the July data providing insights into the non-linearity characteristics of the detector system. The recent DRL release (CPL 4.0/fitsio-compatible) passed basic testing at Garching. It is planned to reprocess the July data through the pipeline system to test the recent bug fixes and QC improvements. SMPs: RGM (VHS PI) reported on the recent ESO SMP letter to PIs; there followed some discussion on the data-products release schedule and the interaction required with survey management and pipeline operations. It was noted that the upcoming UKIDSS meeting at ESO will provide an opportunity to discuss various matters arising with appropriate personnel. 8. ADASS 2007 Nearly all the contributions have been latex'd and seem ok. One more invited talk is being chased up. 67 talks and posters have not been received and hence will not be included in the published volume. 9. AOB Grant review news: MJI noted that the review data for the CASU rolling grant had now been set as 22nd January 2008 and will be held in Edinburgh together with a review of the WFAU grant. Computer systems: MJI and PSB have initiated a general tidy up of the computing enviroment in the main APM area and in the associated store room. Part of this will involve some minor building works to install a partion around the computer racks remaining in the main area to cut down on the background noise. Update plone site: MJI noted that EGS had recently updated the main CASU plone site. Part of the consequences of this was the quiet dropping of the duplicate CASU internal web site. There followed some discussion on this and the overall appearance of the external web site. STH volunteered to feedback constructive criticisms. <<<< APM christmas pary: is happening this Friday (14th) and a small sub-committee of hardened partygoers was set up to make it so. Continuing Actions ------------------ MR advertize illumination correction information MR perform backup verification test using 4 old LTO tapes ALL provide comments on photometric calibration paper MJI review user accounts on APM cluster New Actions ----------- STH check the advertized illumination correction information and update relevant section in calibration paper DWE switch off database server on apm1 and remove related files ALL send in reports for November and December to STH by 3rd January JRL check what is causing hiatus in AAT archive transfers JRL check why pipeline catalogue software and toolkit give different results in a few specific cases JRL draft letter of support to Omar Almaini and Phil Lucas regarding future of UKIRT/UKIDSS and circulate within group STH feedback comments on Plone/HTML websites