1st February 2007 Hoyle Committee Room 10:00 - 12:25 Present: DWE, STH, JRL, PSB, MR Late arrivals: RGM, NAW Apologies: WJS, MJI, EGS Agenda ------ 1. Actions from last meeting 2. Comments on WFAU minutes 3. Recent meetings - VDMT Telecon 4. Data archives update - AAT, ING, WFCAM, UKIRT - data server migration (SYBASE etc) 5. Optical processing update 6. WFCAM - State of raw data delivery 7. VISTA - ESO Calibration Workshop - Surveys Meeting at ESO - Pipeline Meeting(s) at ESO - VDFS notes for SMP 8. ADASS 2007 LOC 9 AOB Minutes ------- 1. Actions from last meeting JRL has finished the fix to the problem with the UKIRT archive (missing header information) STH has not investigated the effects of stacking on the 2MASS/WFCAM non-linearity comparison - ongoing <<<< JRL is still awaiting a reply to his emails regarding the PostgreSQL problems with the AAT archive - ongoing <<<< MJI have emailed the new WFCAM colour equations to DWE and JRL. These have STH now been noted on the Plone technical pages: http://casu.ast.cam.ac.uk/surveys-projects/wfcam/technical/photometry RGM has not finished working on the classification aspect of cross-talk weeding - ongoing <<<< DWE have updated the Plone technical pages on astrometry (cubic radial MJI distortion term (PV2_3): http://casu.ast.cam.ac.uk/surveys-projects/wfcam/technical/astrometry MJI was not here but reported post-meeting that he found out zip about curtaining in the ESO IRACE controllers EGS organized the purchase of the new server as a spare for CASU web services and for running the IPHAS web service. This is now up and running. JRL have released the flat field frames report on the Plone site: DWE http://casu.ast.cam.ac.uk/documents/vista/vircam_flatreport/ MJI have sorted out how to flag header updates MR JRL have released the VIRCAM software and documentation on the Plone site: DWE http://casu.ast.cam.ac.uk/surveys-projects/vista/software/vircam_manual/ PSB raised the issue of trying to keep this up-to-date. DWE was asked to look into automating the updates using cvs. <<<< RGM has acquired the most up-to-date versions of the VISTA public survey proposals. MJI organized an internal meeting during the first week of January to discuss VISTA public survey data requirements (see item 3) ALL paid RWA for the CASU Christmas meal. 2. Comments on WFAU minutes There were too many minutes to read with our current manpower levels :) No comments (some of us had read all of them!) 3. Recent meetings An internal VISTA public survey meeting was held on 5 January to discuss data requirements. JPE also attended this meeting. The main conclusion was that the planned upgrades for VISTA pipeline processing should satisfy the majority of survey users. STH reported on the VDMT telecon held on January 18th. The CASU report for the last quarter was presented with MJI participating from La Palma. The SMPs for the VISTA Public Surveys were discussed. On the agenda but deferred was the purchasing of Paranal computer hardware. There was also a brief discussion about the merits or otherwise of splitting the release of the UKIDSS surveys to avoid the GPS delaying DR2. MJI, JRL, PSB and MR attended the ESO Calibration Workshop (22 to 26 January) and said that it was a very useful meeting. MJI, JRL and MR gave very good presentations that were well attended. Many informative side meetings were also held (see item 7). STH reported on a UKIDSS calibration meeting that was held last Wednesday. This was attended by STH, Paul Hewett, Steve Warren and Nick Cross. There were many action items on STH to fine tune the calibration for DR3. 4. Data archives update AAT: JRL said that a new shipment of DVDs have arrived which takes us up to well within the proprietary period. RWA has started ingesting these and is not finding as many problems as previously. ING: the transfer of data over the Internet is functioning seamlessly. UKIRT: we are waiting for the next shipment. WFCAM: see item 6. Data server migration (SYBASE etc): the data server migration is nearly done. All of the important databases (save one) have been ported. Most of the middleware has also been modified and some testing has been carried out to be sure the search facility is still working. The aliasing of the server will be done in the next day or so. The VizieR mirror will also restart at that time. Very basic backup of the system is being done by JRL. He will formalize it in the next few days. 5. Optical processing update Nothing to report. 6. WFCAM MR reported on the latest 06B data from WFCAM. It was noted that CASU have only received 4 complete nights since October. "We have received a few tapes containing 06B data in the last couple of months. The first two sets of tapes are not completely useful because of two issues: 1) NDR data was not written to tape in an adequate way and it is not possible to use them. The NDR problem mainly effects WFCAM narrow band, Y and Z observations since in general JHK observations are taken CDS read mode whereas NB, Y and Z are taken with NDR read mode. 2) For many nights there are many files missing but it is unclear whether they are completely missing, i.e. they do not exist at all, or they were just missing from the tapes for some "technical" problem. "The approach we took was therefore to try to process what was available and we ended up with the nights between Nov. 29th and Dec. 19th. These are now processed but they still do not include NDR data and hence have not been flagged as ready to copy. "This week we received two sets of tapes with a total of 11 tapes. Four of them contain new data for the nights Dec 21st - Dec 28th. The NDR problem has been fixed so the data for these nights can be fully processed. However: - I've not found any data for the 25th and 26th -> need to check if this is ok. - 27th: chip 4 is missing - 28th: chip 2 is missing "The other tapes contain the fixes for the previous data. In particular they contain: - Chip 1 for: Oct. 26 - Dec. 20 - Chip 2 for: Oct. 26 - Nov. 27 "At the moment I've MEFed all the complete new data (Dec. 21-24) and I am in the processes of dumping to disk the tapes with the fixed chips 1 & 2 hoping to get the remaining chips soon. "A final remark about flat fields: although there seems to be plenty of J,H,K sky flats, there are only a few or none for Y,Z and the narrow band filters. Of course we cannot say what's in the November data because we have not received all of them yet. "By the end of the week we plan also to begin with the data transfers to ESO for the complete nights only (i.e. Dec. 21-24). No data for 06B have been transferred yet to ESO because of the missing NDR data: it made sense to us to begin the transfers only when the all the data was available. Contact has been made with ESO this week to explain the problem and we have not received any reply, so we guess our decision is fine with them." The group were very unhappy about the situation and debated what could be done about this. It was noted that Andy Adamson would be visiting next week and MJI et al. would discuss this with him. <<<< JRL said that he would check the OMP database to see what data we should be getting eg. did they observe over Christmas? <<<< 7. VISTA PSB received the following report from WJS on the VISTA/VIRCAM integration meeting at ATC 2007-01-31: "The camera is in the Prep Room , no visible damage, and its shock recorder maxed at 0.8g during the whole trip, which is good. "The next thing is the fit-check to the telescope, followed by cooldown. We have site power (hooray) , but we're the first to get cut off if there is a problem (has happened once so far). There is some issue that the dome Helium lines haven't been cleaned, so they may have to rig temporary ones , then switch later. Another niggle is the fire-alarm isn't yet connected to Base Camp or VLT so we have to have 2 people (1 isn't allowed ) on the summit overnight to leave power on (!!) - this should get sorted very soon though. "There is a strong chance the Camera will have another cold run in the Prep Lab for much of April/May/June after the He lines get cleaned , and remote login to the camera should hopefully be possible too . (This may not require RAL to go back out again, because it can cool down on coolers-only , it takes ~ 10 days but it gets there in the end). "We then have to await the delivery of M1 and M2. At the ESO Calibration Workshop a (large) number of side meetings were held. In particular a pipeline meeting which discussed the ESO QC system. Apparently, there are two systems: Garching and Paranal. We have been using the Garching model and will need to add a few more bells and whistles to get it to match that at Paranal. There then followed a lengthy discussion on the draft guidelines written by Nigel Hambly for the VISTA Public Surveys SMPs which will hopefully be used as a template. One of the main points was QC and feedback from the PIs. It is very important that CASU gets feedback from the PIs promptly, so that any bugs/funnies get dealt with early on. NAW expressed a worry that we didn't want to be dependent on all these groups to approve the data. Perhaps such a system would only be needed for the early (shakedown) period. The mechanisms for the PIs accessing the data for QC purposes were discussed. It was felt that if PIs want such access then they must ask for it in their SMP. Then suitable mechanism (and funding) can be put in place, if needed, to facilitate this. MR suggested that a system pushing the data onto a PI's home system (along the lines of the automatic ESO WFCAM data transfers) is one possiblity. STH suggested that these guidelines should be written from a VDFS point of view rather than CASU and WFAU. It shouldn't look like two boxes on a pipeline, but one project. These notes should be updated by RGM by Monday and when MJI has returned organize a quick meeting of relevant people. <<<< 8. ADASS 2007 LOC JRL said the plans for ADASS 2007 were progressing well. MR and EGS have been volunteered to do the registration web page. <<<< In particular, information is needed about how to deal with credit card payments. MR was asked to contact Dick Shaw and Paul Aslin to find out more about this. <<<< The venue for the dinner has been sorted out (Globe Theatre), but the caterers still need to be chosen. This will be much less of a problem than finding a venue. JRL reported that some money had been obtained from PPARC to offset some of the costs. There will be a real LOC meeting in the next couple of weeks. <<<< PSB was reminded that he should have been in contact with BT regarding the wireless provision for the main hall. <<<< 9 AOB PSB announced that he will be a grandfather in July - and he doesn't look a day over 21. Continuing Actions ------------------ STH investigate effects of stacking on 2MASS/WFCAM non-linearity comparison JRL repair AAT archive mirroring (PostgreSQL problems) RGM report back on classification aspect of cross-talk weeding New Actions ----------- DWE investigate automatic synching of VIRCAM software and documentation from cvs to Plone. MJI discuss data transfer issues from WFCAM with Andy Adamson next week JRL check OMP database to see what WFCAM data has been observed RGM update SMP guidelines by Monday and organize meeting EGS prepare ADASS 2007 registration web page MR MR contact Dick Shaw and Paul Aslin regarding credit card payments JRL organize ADASS 2007 LOC meeting PSB contact BT regarding wireless provision for ADASS 2007