CASU meeting Monday 23rd May 11:30-13:00 HCR for a change Present: MJI, JRL, AKY, SC, STH, RGM, EGS Apologies: JRF, NAW, MR, JPE Agenda: 1. Actions from last meeting 2. Comments on WFAU minutes 3. Hardware update 4. Meetings and telecons 5. Data archives update 6. WFCAM update 7. VISTA update 8. VST update 9. Spectroscopic stuff 10. ESO in-kind stuff 11. AOB/news 1. Actions from last meeting ----------------------------- JRL - still continues to investigate "rolling" master linearity corrections including "average" channels <<<< STH - was consistent with progress of VISTA calibration papers but will circulate draft versions in time for next meeting <<<< JPE - continue their investigations into predicting region affected by JPF bright star diffraction spikes <<<< SC - reports we might be at quote stage of ordering G-bit switch !! <<<< SC - apm47 and apm49 are now up and running so that ...... EGS - ...... can use one of them as a temporary server for apm25 <<<< SC - home disk space improved somewhat, ancient accounts still need MJI chasing up and removing as required <<<< SC - monitored Blades for sporadic I/O issues -> nada -> ongoing <<<< SC - missing software modules added to apm45 ALL - registered for UKIDSS workshop (well ..... some) EGS - are still rationalising serving up of UVEX data <<<< MJI MJI - finished off March and April WFCAM monthly calibration updates MJI - awaits standdown period for WFCAM before avconf updates <<<< JRF - sorted out the missing processed VISTA night and other odds'n ends JRL - added in extra check for monthly master flats (VISTA and VST) AKY JRF JRF - regenerated mysterious missing images for VISTA 20111124 EGS - dumped and posted "lost" VVV data on pile of LTO4's JRL - has ensured no VPHAS i-band images can make it into fringe frames 2. WFAU minutes --------------- The lower than average transfer speeds have not exactly gone away but may have been related to a lot of local disk traffic here at CASU for both VST and VISTA. We were asked to add the usual couple of extra keywords for VST data aka OSA_TIME= ' ' / OSA time stamp OSA_MFID= 0 / OSA Multiframe ID to the PHUs of images/conf maps. This caused much discussion about how come if you do this with Perl its lightning fast and if you use Python its turgidly slow -> conclusion cfitsio perl modules just adds the extra cards to PHU in the space deliberately left for this; pyfits equivalent must rewrite the entire file. 3. Hardware update ------------------- MJI asked about progress with the 108 Dalmations in the Dog Kennel since he was curious when the surplus LTO4 unit and 30TB array will be available - answer: don't hold your breath. SC and MJI engaged brains and figured out that 3TB does equal 3x10^12 Bytes. 1024^n where n is quite large makes a big dent. As noted earlier the remaining hardware recently acquired, aka apm47 and apm49 is now up and running. STH's transient alert machine did at least hop onto the table but still awaits its Infiniband card -> acquiring timescale clearly related to name There was no clamour for yet more 64-bit software versions and other than the odd gfortran library mysteriously being absent on some machines all is fine with these. Another random disk faux pas occured on apm24_f exciting much rearrangement of disk storage. Two strikes and its out and is now downgraded to storing WFCAM raw data, which is all backed up on LTO4's. It is probably a controller firmware bug that only (so far) manifests when doing rapid updates of flags in subdirectories. Bulk use seems fine so its for raw data then, rather than risking a firmware upgrade. There was the usual MJI whinge about the computer monitoring web pages having a mind of their own for some kit but at least the web address has stayed the same for a few consecutive months. SC will investigate further. <<<< 4. Meetings and telecons ------------------------- The most recent VMC skypecon was on 3rd May but was so noisy that MJI signed out after his bit. Lots of action on the WEAVE, 4MOST and GaiaESO front with more telecons than you can shake a stick at. JRL will be Keele'ing Monday 28th together with Sergey (Koposov) to discuss picky details of spectral processing. The UKIDSS final workshop is next week. MJI, STH, JRF and JRL will be partaking of the culinary delights. Richard Hook visited on Tuesday (22nd May) to discuss PR matters for VISTA/VST. Main topic was how to ensure a continuous flow of good PR material particularly for unusual or novel images/science. One upshot of the meeting was the realtime generation of some VST Carina nebula PR mosaics that had been pipelined in early May, unbeknownst to us. Richard mentioned that the latest VISTA PR image was the M55 pic generated pre-SV !! 5. Data archives update - AAT, ING, WFCAM, UKIRT, VISTA ------------------------ Everything is up to date as usual, and the UVEX suppository and dispenser still needs sorting. The U-band recalibration worked extremely well and now needs propagating to the band-merged products and the whole lot serving through a common portal. <<<< 6. WFCAM update ---------------- Acquiring raw data, ingest and processing are all proceeding apace. There were two slight faux pas, with 20120512 wfcam4 not all being transferred first time around but that was spotted from the wfcam weekly reports, regenerated at JAC, retransferred and reprocessed. The other was the mysterious initial non-appearance of data for 20120504. The WFCAM shutdown period was slightly changed to 23rd May-12th June HST inclusive. At the time of writing these minutes WFCAM processing of May data is finished and data made available to eager punters. This includes the first tranches of UHS J-band observations. The final May calibration updates will be done after the UKIDSS meeting and with any luck so will the <<<< avconf updates noted earlier. The UKIRT Board meeting is scheduled for the 31st May in Edinburgh, closely followed by the UKIDSS meeting. There were various emails discussing the UHS observing strategy and even more discussing Korean involvement and concommitant data release policies. 7. VISTA update ---------------- Receipt of data, ingest, checking and processing continues to go smoothly since the switchover to Internet-only transfers. The process of catchup for VISTA processing has almost happened, helped by the 2nd-5th May VISTA technical downtime for replacement of the M2 hexapod leg. The release schedule is bit further back but both March and April are almost ready to go, with March in particular bulging with VVV Ks-band data. VVV, Chile Branch, were sent the entire VVV dataset on LTO4's courtesy of Eduardo's hard work. Hopefully this time they'll keep the backup copy. Eammon Kerin's Ks-band images, confidence maps, and sky frames from 2009-> end of March were also dumped, this time to 4TBs of USB disks that he had shipped to us and were subsequently dumped to and shipped back to darkest Manchester. Let's hope that this is the last, for now, of these unwanted extra jobs. Whilst doing the final month of these (unofficial March) MJI noticed a missing sky frame v20120321_00164_st.fit 179.B-2002(C) d053 Ks sky_20120321_158_Ks.fit ...... copying v20120321_00164_st.fit ...... ...... copying v20120321_00164_st_conf.fit ...... ...... copying sky_20120321_158_Ks.fit ...... cp: cannot stat `sky_20120321_158_Ks.fit': No such file or directory and is curious to see if this gets picked up during final checks. JRL noted that he had found some "detached" sky frames lurking in the pre-release directories and will see if that one is among them. <<<< Finally, RGM noted evidence for systematic global differences between VHS, ULAS astrometry and ICRF of the order 100mas in RA and Dec - summary included for reference in full below. The offsets in mas are: dRA dDec VHS-ICRF 107+-5 94+-4 LAS-ICRF 100+-4 90+-4 SDSS(DR7)-ICRF 52+-1 51+-1 For ICRF he used a compilation of VLBI radio source positions from http://astrogeo.org/rfc/. This is larger than the ICRF list at http://hpiers.obspm.fr/icrs-pc/ 2MASS is epoch circa 2000 and we are now 10 years later so we might expect some low level bulk proper motion induced differences. Of course there may well also be differences in global 2MASS accuracy at this scale. Using UCAC3 proper motions, Sergey Koposov plotted the expected offsets in mas for 10 year time difference for the southern sky (only, because in the north UCAC3 proper motions have problems). The offsets are in qualitative agreement with the plots made by Sergey and show large scale variations across the sky due to the direction of the Solar motion. see http://adsabs.harvard.edu/abs/2010AJ....139.2440R UCAC4 will be released this year sometime, aka ...... "the public release of UCAC4 is scheduled for June 2012". http://www.usno.navy.mil/USNO/astrometry/optical-IR-prod/ucac When it is, we will, investigate ...... further, that is. <<<< 8. VST update -------------- VST data receipt and ingest is proceeding smoothly. The Chilean u-band OBs have started being taken in the ATLAS fields and we need to make sure permissions are set to enable us to get the data here seamlessly. <<<< For the interim at least the plan is to process these as normal and not worry about combining them with the regular ATLAS u-band data. For now the list-driven (from g-band) catalogues from these will be extracted independently at the band-merging stage and the catalogue results combined. Monthly release of VST data will follow the VISTA model now that the fixed catalogues are available. This constitutes v1.0 of VST releases and with immaculate timing now also includes the ESO OB grades. We decided to leave out any colour equation tweaks and zpt tweaks for now and will wait for the final version of APASS (~end of the year) before revisiting these. This is in part due to our plan to use APASS, which is on the SDSS AB system, to include an alternative AB mag zeropoint in the headers, plus possibly also as an independent check/update of the less frequent SA-based zero-points. The one VST foible not yet addressed, are the random gain changes in some of the detectors. Independent ab-initio fixing of these robustly is not possible but may be feasible at the band-merging stage, particularly for VPHAS, though ATLAS will be more challenging. As apparent from the previous minutes, CASU sent off a contribution to the VPHAS and ATLAS PI reports for the P90 OPC meeting. One of these was the total no. of fields observed for ATLAS which not including duplicates, EGS makes to end of April to be: * Number of fields in ugriz: 345 * Number of fields in gri (including the above): 519 * Number of fields in u,g,r,i,z respectively: 954,921,1180,1224,1280 The mismatch between the totals is a little surprising suggesting a somewhat more chaotic observing pattern than we would have expected. 9. Spectroscopic stuff ----------------------- Yet more meetings and telecons (GaiaESO, 4MOST, WEAVE) as noted earlier and continued steady progress on the GaiaESO processing, data formats and ICDs. JRL has been shovelling GaiaESO data through the CASU and ESO pipelines, keeps finding new ESO pipeline foibles, and comparing results (and formats) with the Keele pipeline. Who knows, we may even soon converge on a filenaming scheme. Picky details such as rv accuracy at the ~1km/s scale are now coming to the fore so presumably everything is going well. EGS continues to burn the midnight oil analysing simulated spectra for 4MOST and trying to shoehorn requirements, FOMs, and science goals into a coherent story. Hopefully now that the ESO meeting to discuss possible telescopes has come and gone this simulation craze will become more focussed and less "because we can". 10. ESO in-kind stuff --------------------- All of the HAWKI and VIMOS imaging data is sitting there in eager anticipation of the release of version 6.1 of the ESO reflex workflow to process it. JRL and MJI had a telecon with ESO folk to discuss progress on the in-kind work and to deliver a verbal report for inclusion in the report to ESO Council. A slight more formal quarterly report is due mid-June. <<<< JRL has already incorporated the CPL version 6.0 changes for the standard ESO VISTA software deliverables. 11. AOB/news ------------ One day we'll have time for the choice CASUhelps, but this week its Cambridge Beer festival week - d'oh ! The blinds for the select APM upper (opening) windows duly arrived and were fitted, almost without fuss. The rubbish opening mechanism ones were fine, the greasy pole opening one declined to be blind-ridden and opening/closing at the same time. However, the latter soon yielded to pursuasion with an improvised screw-driver based chisel, late one boring afternoon. The second (spare) photograph of PSB, beer mug in hand, will be mounted on the wall of the Upper Landing of the Corfield wing and join a few others who also departed us too soon. Finally, Marco's dad gets the CASU prize for best contribution to work output in recognition of the new expresso machine Marco returned with from a Gaia meeting in Bologna .... *fizz* Continuing actions ------------------ JRL - investigate benefits of a "rolling" master linearity correction or even an average channel one STH - progress evermore the VISTA calibration papers writeups and circulate draft papers before the next CASU meeting JPE - finish off investigation of VISTA diffraction spikes by defining JRF affected radius -v- magnitude SC - keep chasing up acquisition of a new G-bit switch SC - sort out ancient login accounts MJI SC - monitor for sporadic Blade I/O problems EGS - rationalise serving up of UVEX data MJI MJI - organise av conf column updates for WFCAM standdown period New Actions ----------- EGS - use one of apm47 or apm49 as a temporary server for apm25 SC - see if mysterious monitoring web page issue has a solution MJI - do May WFCAM monthly calibration updates and release data JRL - see if detached sky frames need a new home MJI - keep an eye out for UCAC4 and assess when it is out EGS - make sure we get permission to copy Chilean u-band ATLAS data AKY MJI - produce quarterly report for ESO in-kind stuff