CASU meeting Wednesday 25th September 11:30-13:00 APM Present: MJI, AKY, STH, JRL, GM, EGS, CGF Apologies: MR, NAW, RGM, JPE Agenda: 1. Actions from last meeting 2. Comments on WFAU minutes 3. Hardware update 4. Meetings and telecons 5. Data archives update 6. WFCAM update 7. VISTA update 8. VST update 9. Spectroscopic stuff 10. ESO in-kind stuff 11. AOB/news 1. Actions from last meeting ----------------------------- Agenda item: progress on ALL papers - meetings mid-monthly will resume again STH - detailed analysis of all the VVV Bulge and disk fields is ongoing to MJI examine the extinction over the entire region. Exinctions maps have CGF been generated and other byproducts will be updates of the extinction calibrator coefficients and the zero-point offsets - ongoing <<<< JRL - replacing apm5 Sybase server proceeding apace albeit slowly <<<< EGS and we learnt that there are more databses on here than expected MJI - VST dome flats-v-twilight flats keep watching this space <<<< SC - is ordering another fibre channel raid for apm49 new purchase ordering system notwithstanding <<<< SC - has been investigating apm41 and carried out firmware updates almost but not quite finished off <<<< MJI - sent an updated version of the APM hardware usage to STH who has cunningly hidden it on the Plone internal web pages EGS - has added a warning viz these Rice-compressed files may contain Rice compressed data to the email sent out to ING archive users EGS - and investigated adding an option to add .fz on-the-fly the implementation of which will be deferred until demand stampede MR - changed 13B web pages for WFCAM to 13a to keep the semester extension separate but politically correct 2. WFAU minutes --------------- No comments 3. Hardware update (and software) ------------------ The UKLight connection between the IoA and UCS will change to using the new fibre at 10:30am this Thursday. JANET are planning to migrate all remaining Lightpath circuits to new equipment on the New Museums site on Thursday 26th September at 10:30. In the itermim the IoA has already initiated the relevant changes at this end so we can guarantee that nothing will work after the changeover (subsequently verified by events). Amazingly nothing stopped functioning after all the recent changes in what files go where on assorted recently acquired hardware. SC and MJI will investigate encore une fois the usage of home disk space and attempt to remove some Jurassic-era users. <<<< 4. Meetings and telecons ------------------------- Uncertainty over when the VMC monthly telecons are occuring is the excuse this month. MJI attended the first day of the EGAPS meeting in Warwick 5-6th September and presented a VST VPHAS processing and calibration update. STH did his bit for our CHEOPS involvement by attending a meeting in Marseille, presenting our ideas for the quick-look pipeline. Amusingly les francais n'etaient pas amused. NAW did his bit by helping organise the recent LSST meeting held at the IoA. Much interesting discussion ensued. The UK 4MOST participation bid had its PPRP grilling and later a site visit and now awaits the verdict. ADASS is next week and JRL, EGS and NAW drew the short straws, and the next relevant WEAVE meeting is at the IAC toward the end of October. NAW and EGS will attend. 5. Data archives update - AAT, ING, WFCAM, UKIRT, VISTA ------------------------ All are up to date and running smoothly. 6. WFCAM update ---------------- WFCAM is back from its August break. A new set of flats were taken and are being used for the last week or so of August data and all the September processing. The first few nights of August were processed with July flats. Data releases have caught up with processing and data up to end of July has been flagged for download. However, the most notable event is Thor's upcoming retirement on the 27th. We contributed a collective thank you from CASU and wish him well in his retirement. The ADASS meeting provides a suitable chance to find out more about WFCAM (and UKIRT's) future and JRL together with Bob Mann will see what they can glean. <<<< 7. VISTA update ---------------- Two more months of VISTA data have just been released, thus all the VISTA data obtained up to 31st of July 2013 are now fully processed and fits files are available for download from CASU (http://casu.ast.cam.ac.uk/vistasp/). Eckhard sent over a short list of minor hiccups which have all been dealt with, including a completely missing night of data, 20130518. This latter was absent on both raw and processed since, unusually, no data at all had been taken that night. There was one catalogue file with a bunch of NaNs in some flux entries but as these were all edge-effect spurions they were all nullified instead. There has been a brief confusing flurry of emails regarding VIKING reprocessing at Terapix. To cut a long story short this request stems from a desire to recombine VIKING data into mosaics to generate a seeing-matched set of images for photo-z work. There is also the usual gripe about the lower surface brightness outer regions of galaxies not being well measured in the standard CASU output. Both fine aspirations but quite why these might involve redoing the sky subtraction and pawprint stacking/mosaicing a la UltraVISTA is beyond us since you could clearly start with the already stacked and fully calibrated pawprints and take it from there. Meanwhile the recently published UltraVISTA data paper is less than honest with the statement "VIRCAM images are transferred to the Cambridge Astronomy Survey Unit (CASU) for pre-preprocessing and removal of the instrumental signature". This is somewhat misleading as VIRCAM images are transferred to CASU for full scientific processing and astrometrically and photometrically calibrated processed images and catalogues are delivered for all VISTA data, including UltraVISTA. The fact that UltraVISTA then unpick the sky subtraction and stack multiple nights with suitably deep object masking is another story. Interestingly we also learned that UltraVISTA remove all sign of CASU processing from the images. This includes versions of software used and release version, or dates, so there is no way of tracking back through what has been done. In response to a query from Bo Milvang-Jensen regarding how bad pixels are defined we reproduce for posterity JRL's reply. "The procedure for defining bad pixels involves looking at the linearity series. After dark correction, linearisation we form a mean flat. Then each input flat in the series is divided by the mean flat. If a pixel is discordant on a number of the flat corrected images, then it's marked as bad. So it really doesn't have anything to do with whether it's got high or low QE. It's simply a case of whether its QE is consistent with time and ambient flux" ..... and is hence predictable or not. No news about Phase3 news - must be good - though as part of our VVV extinction investigation we noticed an astro-ph submission (1308.0593) which had actually managed to download VVV's Phase3 DR1 data from ESO and make scientific use of it ! 8. VST update -------------- VST data receipt and processing is up to date. Releases are a bit behind but as soon as September is finished processing will undergo some catchup. Giacomo Beccari has been attempting to use our catalogue dumper, fitsio_cat_list to create ascii versions of some VPHAS ESO Phase3 catalogues ! Somewhat caught by surprise we had to figure out how to cope with all the FITS headers changes that ESO enforced on the catalogue deliverables. No doubt someone else will attempt to do the same for VISTA and expect the software to magically make sense of the exposure time changes, among other things that ESO demanded for VISTA data deliverables. We will no doubt be receiving a call in due course. Mike Read lobbed over an interesting grenade regarding X-Y catalogue positions of unmatched sources from ATLAS g-band data. This had been sparked by a query from Nigel Metcalfe on this topic. Apart from ripples to the left of them and ripples to the right, there was also a curious spike at around 3 pixels in the underexposed dither regions at each end. As did Mike, we surmise it is down to edge effects and possibly caused by the fact that we trap off-frame coordinates of spurious object to lie within the nearest image edge - it is possible to end up with weighted 1st moments that can take you off-frame. The near 0 edge is probably more obvious since the other end edge is more varied. There is also an issue with the outer edges of the 4 corner detectors generating more rubbish than usual due to vagaries of the scattered light from the masking strips. Finally, just as for VISTA, all is currently quiet on the Phase3 news front. 9. Spectroscopic stuff ----------------------- Gaia-ESO: a minor faux pas with the stacking of GIRAFFE spectra was identified and subsequently fixed. As a consequence iDR2 was re-released to the WG leads as iDR2.1 with a few other minor fixes also included. AKY has continued the testing of and griping about the ESO-Reflex workflows and has delivered said gripes to ESO for feedback. Just as in the previous minutes all effort on WEAVE and 4MOST has been spent preparing for yet more discussion/meetings. No further news about MOONS. 10. ESO in-kind stuff --------------------- Most of the effort over the last month has been spent preparing for the v0.5 release of HAKWI processed data. In doing some of the final checks a few further improvements to the software have been required. All the HAWKI data have been reduced noch einmal with an updated version of the pipeline software (v0.53) and even this is not quite the current version (v0.55). GM has gone through the full suite of processed HAWKI data and catalogued the fatal and non-fatal errors in the processing. There are still a number of issues to be fixed in a later version of the pipeline. The gain of the HAWKI chips has been evaluated over 4 years of data and some interesting patterns of the gain vs. time noticed. The potential origin of these features is being investigated before finalising a convenient look up table. The usual post-processing checks and QC diagnostics are now being generated and these will need propagating to the status web pages. <<<< 11. AOB ------- A few folk took the hint and have signed up for University Sports Centre Membership. Continuing actions ------------------ STH - compute the ZP offsets for the VISTA calibration for y,z-band and MJI update the extinction calibrator coefficients as necessary CGF JRL - keep reporting on further progress with retiring our apm5 Sybase server EGS MJI - are the VST dome flats that stable and how do they cf. to twilights SC - order yet another fibre channel raid array aka apm49_b SC - finish off investigations into apm41's misbehaviour New actions ----------- SC - revisit yet again usage of home disk space and ancient accounts MJI JRL - find out more about WFCAM and UKIRT's future at ADASS meeting EGS - propagate the HAWKI processing info to the status web pages