Wednesday 27th October 2010 APM Meeting tables 11:30 - 13:10 Present: JCR, MJI, STH, EGS, JRL, AKY, NAW, JPE, SC, RGM Apologies: MR Agenda ====== 1. Actions from last meeting 2. Comments on WFAU minutes 3. Meetings and Telecons 4. Data archives update - AAT, ING, WFCAM, UKIRT, VISTA 5. Optical/NIR processing - HAWKI, INT WFC, MegaCam, Subaru, VST 6. WFCAM update 7. VISTA update 8. CASUhelps 9. AOB Minutes ======= 1. Actions from the last meeting -------------------------------- STH - finish redefining colour equations for VISTA and when done update MJI calibration plan document - err ..... yes <<<< EGS - put WFCAM illumination correction tables on the web <<<< MJI RGM - testing of catalogue software still ongoing. MJI noted that it is now being routinely used by other VISTA PI teams <<<< JPE - suggested that the current links to the ESO web pages for the PR images are sufficient and as there was no great enthusiasm for more verbage this action was dropped JPE - signing off the pipeline deliverables is a tad more complex than it possibly should be and although desirable is not suitable for an action item since it is way out of our control EGS - talked over the SV v1.0 data with the respective SV teams during the MJI VST meeting visit. Final delivery of the catalogues is stalled awaiting the colour equation updates - ongoing <<<< AKY - looked into various aspects of the ellipticity trends and sent results to Marina at ESO for discussion at the IOT telecon STH - emailed the Carina team about the HawkI results and commented on their estimates of the calibration errors MJI - WFCAM calibrations in regions of high extinction investigation is STH still ongoing <<<< EGS - devised cunning VST pipeline plan and presented it at ESO VST MJI review meeting JRL - added NICOMB keyword to headers to specify how many dithers went into each stacked pawprint in each extension (even though it is possible to figure this out from the provenance information). MJI noted that he needs to update the summary tables total exposure time column to account for the occasional missing dither component. <<<< EGS - has set up a 'known features' page which is already proving useful http://casu.ast.cam.ac.uk/surveys-projects/vista/technical/known-issues 2. Comments on WFAU minutes --------------------------- capisco capisci capisce capiamo capite capiscono 3. Meetings and Telecons ------------------------ The VST Review meeting was on 28/29th September. MJI and EGS attended for CASU and presented a cunning pipeline plan and also demonstrated some equally cunning software running on "similar" optical data. Astrowise and VSTube also presented their proposed processing systems. The outcome was very successful for the PI surveys Atlas and VPHAS (and KIDS), with each being approved, even with a few extensions/revisits of target survey areas factored in. They were also given higher priority than the remaining GTO programmes. The talks presented are available at the following web address http://www.eso.org/sci/observing/policies/PublicSurveys/vstreview.html but you'll have to ask nicely for the uid and password (vst publicsurveys) VST astrometric distortion is expected to be low so if born out by reality that may simplify some of the processing particularly for the deeper surveys. It is still unclear how the VST data will get to CASU or even if we only get to see the UK PI subset of it. The schedule is fairly agressive with VST 1st light Dec 2010, integration and alignment by the end of Jan 2010, and commissioning -> late summer 2011 -> ESO acceptance. If all goes well surveys commence around August 2011 so we need to be ready for then. At this stage it is not clear if we will be able to get access to some of the commissioning data prior to surveys starting, but we can revisit this issue further downstream. We gleaned a few useful numbers: 15 micron pixels = 0.21 arcsec, 2k x 4k EEV 44-82s x 32 and usual MegaCam-like arrangement. The faulty CCD has been replaced and the whole array is co-planar to +/- 20 microns. Flexure between the AG and the science CCDs is ~3 microns (i.e. ~40 mas); r/o time is ~30s; filter changes take ~30s to 2 mins depending on where in one of two 6 filter stacks the next filter is. As with VISTA, at this stage it is unclear which operations (e.g. slewing, r/o, filter change, AG setup) can be parallelised and hence what realistic overheads are. The monthly VISTA IOT took place on 5th October. AKY presented the ellipticity measurements and our assessment of the QC status, which lead to some discussion about the accuracy of the assigned ESO OB status and what, if anything, to do about it. Upcoming events include: a VMC telecon on the 28th October for which MJI will listen in; ADASS starts on the 7th November, MJI and JRL are attending; Tom Shanks is organising an Atlas meeting in Durham on 23rd November, MJI and EGS will go see what's up; the ESO Phase III workshop will be on 30th November an all-ticket event only, EGS will be the CASU rep in the guise of a VHS accolyte. There has been a lot of recent activity in Eulicd, Plato and assorted GAIA-related ground-based stuff - way too numerous to mention here. 4. Data Archives ---------------- AAO, ING, WFCAM, UKIRT, VISTA et al. all trucking along fine. 5. Optical/NIR Processing --------------------------- Everything is in order and nothing noteworthy to report other than some long-standing CTIO Mosaic II data was processed, finally. EGS mentioned that he also had some of this to deal with and wondered if there were any extra gotchas - you betcha. 6. WFCAM Update --------------- All available 10B data has been received as expected and has been ingested into the raw data archive with the appropriate bytes transferred to the ESO archive. Processing is ticking along nicely once the full set of monthly flats was sorted. Two lucky punters have asked for the raw data in order to do their own processing, we wish them well, and there have been the usual gamut of early access requests dealt with as they come in, together with yet more dribbling requests. As the dribble pool sometimes laps at our feet JRL has been tasked with CASU-tooling it, the equivalent of removing the drain plugs in the floor of MkI Mini's after driving in heavy rain. <<<< The September release is a tad behind its normal schedule but will be kicked out of the door in a day or so. Meanwhile, the Cloudshine project reprocessing (from 2005b, 2006b, 2007b data i.e. pre-offset sky strategy in place days) is going well. There are about 14 part-nights of this to deal with, and this should also be finished in a few days. This project requires a "reject sky" doing manually but once setup runs using the normal pipeline. MJI wrote and submitted a UKIRT board report by the 22nd October deadline. This is for the November meeting to be held in Oxford. He also sent in a revised estimate of the cost model for CASU processing. Whether or not this is linked to the note to the UKIRT Board on "Options for an extended UKIDSS programme" sent in by Andy Lawrence remains to be seen. 7. VISTA Update --------------- A new disk arrived, even as we met, containing data up to and including the 13th October. Processing-wise, September and early October have been completed (up to and including 6th October). However, whilst processing the data from the 5th and 6th October, JRL spotted a problem with the upper half of the array (detectors 9-16 read out by two IRACE controllers in the same Gas Mark IV box !!) that rendered most of the data useless. He is currently iterating (or was that irritating) with ESO on the cause and extent of this. Results for October 7th -> are eagerly? awaited. Slightly worryingly the ESO QC checks had essentially missed this problem. There have been no updates on ESO QC re-grading. When and if this happens, it will involve a header update. Hopefully this can be done in parallel with a few other minor header updates viz-a-viz tweaking the astrometric and photometric solutions. As explained in the last minutes, Omar and the UDS team noticed an interesting stripey pattern in J & K-band deep stacks in the WFCAM UDS data prior to DR8 release. It is worth doing our own pre-emptive strike to look for similar problems in say the season#1 UltraVISTA data too e.g. by stacking stacks. Although no obvious problems were visible in the deep SV data this was hardly pointing in a good place to spot low-level subtleties. A spritely UltraVISTA season#1 should help to flush out any low-level gremlins and may even be useful for science. AKY and EGS will make it so <<<< The VIDEO emporium has been asking about the format of object masks to be used in sky estimation (tile level, 8-byte FITS image, binary 0 or 1 pixel values, with WCS - if you are curious). We are guessing that shortly thereafter <<<< delivery of some correct ones a reprocesing request will follow. MJI has been attempting some science with VISTA VHS data of the Carina dSph, cunningly positioned at the junction of 4 separate tiles, and re-iterated the problems caused by having 96 different PSFs in each tile. Even with aperture photometry the need for 96 different aperture corrections around the tile causes a "blurring" of the photometry compared with the pawprint component parts. Although this bum-biting problem was anticipated years back it still has teeth. To rub it in, RGM and EGS had in parallel been investigating band-merged tile products for the VHS and had even noticed multiple stellar sequences in some fields due to the seeing varying by almost a factor of two in some of the component pawprint parts of tiles. It appears that VHS is probably worse off than other surveys since it is a.) shallow and b.) taken in worse than average seeing conditions. Approximately 20% of the band-merged VHS tile products investigated had a serious problem with the photometry due to this effect (though it is a moot point if most of this data passes the QC requirements). Although this issue has been presented at various talks by CASU folk, and there is even a bit of spiel about it on the web pages, it is still not widely appreciated. Mulling caps have been donned - watch this space. JRL notes that the CRVAL problem seems to have been almost entirely cured by a Paranal software change that allows the autoguider more time to settle down before beginning exposures. Short term this increases the overheads slightly, long term its gotsta be betta. There was an impressive ESO Photo Release eso1039 - VISTA Reveals the Secret of the Unicorn - of our green scarab beetle picture, though curiously there was no beetle to be seen in their version. MJI and EGS got tired of mis-reading what the roles of CASU and WFAU are in assorted VISTA articles and added a few words to clarify this on the VISTA Data Access pages. A VMC survey paper has been doing the rounds, and assorted draft progress reports to ESO have also been landing in the inbox. The ESO messenger contained an upbeat VVV survey article but in spite of noble attempts to boost the transfer bandwith to Santiago, the next lot of VVV data will need to be ferried via disks as before. Handover will probably happen at the Phase III workshop and shortly thereafter Maren Hempel will be visiting to discuss processing issues. 8. CASUhelps ------------ There have been a wide range of CASUhelp submissions this month including: a misunderstanding about aperture corrections - ahh so that's why you need them; a request for help in calculating limiting magnitudes - 5-sigma; why does fitsio_cat_list produce infinite magnitudes occassionally for a subset of tile catalogues - answer 2.5*log10(flux/0.0) -> Jim'll fix it; is the WFCAM processing optimal - hmmm ....; spikey PSFs for WFCAM interleaved data - dribble dribble dribble .....; and an enterprising PhD student from QMUL has produced some interesting VISTA VIKING -v- UKIDSS LAS comparison plots easily winning the monthly prize for the best question -> yet more mulling required. The most amusing question of the month was not directly asked but was part of our feedback on a draft paper. Horizon's are a bugger. 9. AOB -------- The new VISTA storage space system - a Supermicro - apparently is sentient, i.e. it has a mind of its own. SC is working on it, meanwhile, there is space available on apm19_e as a temporary bodge. EGS reported a series of access attacks on the CASU web server over the last weekend causing it to run rather slowly. Blocking access to the IP address causing them seems to have cured the problem but needs further monitoring to see if it recurs. To finish on a high note (Allegri's Miserere with the extra flourishes ?) the CASU+Admin Christmas lunch is on Wednesday 15th December. The assembled hordes were asked to verify, or not, their attendance by emailing MJI. <<<< Continuing Actions: =================== STH - produce "final" VISTA colour equations and finish revising MJI calibration plan document EGS - put WFCAM illumination correction tables on the web MJI RGM - finish catalogue software tests EGS - SV v1.0 catalogues paused awaiting the above colour equations MJI MJI - finish WFCAM calibrations in regions of high extinction STH New Actions: ============ MJI - update summary table software for total exposure time mod JRL - export FITSIO_DRIBBLE to the CASUtool box AKY - stack the stacks for season#1 Ultra VISTA data to check for features EGS JRL - email details of howto present object masks ALL - let MJI know if you plan to attend the Christmas lunch.