CASU meeting Wednesday 22nd January 11:30-13:00 APM Present: MJI, NAW, JRL, AKY, GM, EGS, CGF, SC Apologies: MR, STH, JPE, RGM Agenda: 1. Actions from last meeting 2. Comments on WFAU minutes 3. Hardware update 4. Meetings and telecons 5. Data archives update 6. WFCAM update 7. VISTA update 8. VST update 9. Spectroscopic stuff 10. ESO in-kind stuff 11. AOB/news 1. Actions from last meeting ----------------------------- Agenda item: progress on ALL papers - nothing much has happened on this front due to unexpected retrenching. STH - further detailed analysis of VVV Bulge and disk fields is ongoing to MJI examine the extinction over the entire region. Better extinction maps CGF have been generated and analysis of the extinction calibrator coefficients and the zero-point offsets. STH was absent so ongoing <<<< JRL - no further action on the apm5 issue on hold for now while other EGS things take precedence <<<< MJI - VST dome flats-v-twilight flats also on hold though new baffling arrangements make it worth rejuvenating <<<< GM - are composing a FAQs page or two and will present an update on CGF progress as the next meeting <<<< STH EGS EGS - is still investigating how the seeing=-1 QC check information is used as part of our pre-release checks and what happens to it <<<< MJI - RAID array for apm49_b was installed and even works SC - Blades are currently all working but ........ MJI - forwarded the address of the network monitoring page MJI - WFCAM release for October got delayed due to other events <<<< 2. WFAU minutes ---------------- Many thanks to WFAU, and Eckhard in particular, for demonstrating that the VDFS catastrophe data recovery plan worked. Tightening up of security means a revised modus operandi for adding in the successfully copied flags for processed data and a more restrictive access for plone DQC logins. There have been a few emails regarding updating the ESO grades for VST and VISTA data due to the ESO habit of regularly updating these things. WFAU will no doubt be pleased to know that EGS will send a text file list with the files and grades rather the pdf's we get from ESO. <<<< 3. Hardware update (and software) ------------------ The RAID array aka apm49_b was installed and has been tested to destruction. There was a brief discussion about software versions and updates on the supermicros. One of these (apm48) will be used for update tests prior to <<<< killing off the entire lot. 4. Meetings and telecons ------------------------- The VMC monthly skypecon was on Monday 21st January from 1-2pm and MJI managed both to remember and also bring his laptop. MJI gave a brief report on the status of data processing for VISTA and the data recovery that was required post 29/11. There will be a VMC science meeting in Edinburgh the week commencing May 19th for those interested. There is an upcoming WEAVE meeting at the usual spot near Schipol. NAW and EGS will do the honours. Biweekly OU-VIS Euclid telecons have to sprung to life with assorted updates required for the Validation Process for the various processing workpackages. Rumour has it a VISTA IOT meeting is on the cards in the next few weeks - watch this space. 5. Data archives update - AAT, ING, WFCAM, UKIRT, VISTA, VST ------------------------ The archives are up to date and (mostly) also backed up on LTO4 tapes. As the VISTA raw data backups were getting a bit behind GM was asked to take over and bring them up to date. <<<< UVEX processing of earlier seasons data is stalled awaiting some human CPU time. EGS noted that he was still copying over (amazingly slowly) the Subaru raw imaging data for processing. 6. WFCAM update ---------------- There have been the usual assorted requests for early access to the last of normal operations mode WFCAM data. Processing of that is completed but October, November and December await the final calibration update prior to release. <<<< There were a few minor hiccups switching to transferring and ingesting data from the new operating mode, mainly related to the larger than usual numbers of files. These issues have been sorted and raw data transfers and processing are on track. The back catalogue of reprocessing and/or fixing requests has started growing again with a request for more nebulising from Phil Lucas. All of these have been on hold until all processed WFCAM data has been recovered and checked. 7. VISTA update ---------------- We have just released two more months of VISTA data so all the VISTA data obtained up to 31st of October 2013 are now processed and fits files are available for download from CASU (http://casu.ast.cam.ac.uk/vistasp/). Processing of November is almost complete bar one recalcitrant tile. As of this month we have added network IP access restrictions for logging in to the above plone web page (though login is only required to copy data files). For login users now have to have their network IP address added to the list of allowed domains. This restriction will be reassessed each month to see if the benefits outweigh the disadvantages. A VISTA photometric ZP long-term variation update is due, the last one covered up to 201304. EGS will make it so. <<<< Eckhard reported a few broken pawprint catalogue files. These appeared to be due to a few rows of extra garbage in one (random) extension so were fixed by garbage dumping rather than regeneration. 8. VST update -------------- Ingest and processing is pretty well up to date. Releases are lagging behind but will be kicked into gear by the next meeting. <<<< ESO sent out a "dear VST Public Survey team" email informing that modified sky baffles were installed at the VST on January 8th 2014. Claiming that "this shall significantly reduce the effects of straylight, in particular during bright moon periods, and hence greatly improve OmegaCAM science observations as well as the flat field quality". Additional modifications to the baffles are expected to be performed in February which may again lead to a change in the flat field morphology. We await the results with interest. EGS kindly sent the ESO User Support team a copy of his script for automatically downloading raw VST data to help non-public survey PIs maintain their sanity. Nigel Metcalfe sent us a list of his derived first-pass colour terms for VST Vega with respect to SDSS AB. These are more or less as expected namely u_sdss - u_vst = 0.01(u-g)_sdss + Cu g_sdss - g_vst = 0.04(g-r)_sdss + Cg r_sdss - r_vst = 0.03(g-r)_sdss + Cr i_sdss - i_vst = 0.005(r-i)_sdss + Ci z_sdss - z_vst = -0.05(i-z)_sdss + Cz The constant terms vary from field to field depending on how good the ATLAS calibration is, but very approximately they are Cu~0.8, Cg~-0.1, Cr~0.1, Ci~0.4 and Cz~0.5. In parallel we have begun a trial adding new zeropoints to the "fix"ed catalogues for 2011 amd 2012 data and are gradually working our way through 2013. In addition to the existing Vega zeropoints based on the ESO stds fields observed each night we have now added zeropoint information based on APASS. These are only AB-like magnitudes since the colour equations to convert to the VST system do not guarantee that the actual zeropoint is exactly on an AB system. It is not trivial to find the best way to use them since for some fields with few APASS stds the "rms" error on the zeropoint edges 2-3%, not counting residual systematics in APASS. To mitigate against this we also provide the average nightly zeropoints computed using APASS. Obviously this latter includes all processed data not just ATLAS and VPHAS. An example randomly chosen fixed catalogue header for each extension now has the following magnitude zeropoints information present ............. MAGZPT = 23.754 / Photometric ZP (mags) for default extinction MAGZRR = 0.04 / Photometric ZP error (mags) EXTINCT = 0.05 / Extinction coefficient (mags) i.e. the original Vega one based on ESO std fields observed each night APASSZPT= 24.147 / APASS ZP (mags) for default extinction APASSZRR= 0.012 / APASS ZP error (mags) APASSNUM= 104 / Number of APASS standards used NIGHTZPT= 24.155 / Average APASS ZP (mags) for night NIGHTZRR= 0.019 / APASS ZP sigma for night (mags) NIGHTNUM= 45 / Number of ZP used in band for night and the new batch. Things that users might want to try are: use APASS per file as is, use nightly APASS, use a combination if say individual APASS estimated error or number of stds is not good enough switch to nightly - it needs some experimentation on large survey areas to see which yields the most useful results. The example above happened to be KiDs i-band and the difference between Vega and AB is about what you would expect. 9. Spectroscopic stuff ----------------------- Gaia-ESO: JRL has written part of a draft of the Giraffe data reduction paper and has passed it on to GG so that he can add his contribution. JRL has also spent some time digging into the guts of the GES giraffe data to work out the issues that exist regarding background correction. He has written up a small internal paper outlining his findings and has circulated this to the WG heads within the GES consortium for comments. Once he has received some feed back this will be incorporated into the GES giraffe data reduction paper. WEAVE: the usual round of discussions/meetings and document/papers updates continues. 4MOST: we are still involved in 4MOST preparations involving throughput simulations and overall design of data reduction system and are also helping out with the science operations concept document. MOONS: nada to report 10. ESO in-kind stuff --------------------- Just before Christmas ESO sent a report with feedback on the pipeline software. By and large the comments are positive, but there is still a lot to be done. A report on the first data release (roughly 10TB of HAWKI and VIMOS imaging data) is expected at the end of February. The processing progress web pages for HAWKI are up and running albeit limited to interal access. An equivalent for VIMOS is being developed. VIMOS photometric calibration and QC still needs some work. AKY continues to progress the workflows for HAWKI and VIMOS and is investigating visualisation schemes based on a python interface. GM has almost finished compiling a list of astrometric standard objects for improving the astrometry of the HAWKI and VIMOS fields. The list comes from VISTA, WFCAM, or VST. JRL will test these out shortly. <<<< If the tests are successful GM will finish off these supplementary catalogues. 11. AOB ------- On Tuesday the South African Science Minister and his entourage of 20+ people dropped by as part of an almost surprise visit during a tour of the new Battcock Centre and the old IoA. Continuing actions ------------------ STH - compute the ZP offsets for the VISTA calibration for y,z-band and MJI update the extinction calibrator coefficients as necessary etc.. CGF JRL - keep reporting on further progress with retiring our apm5 Sybase server EGS MJI - are the VST dome flats that stable and how do they cf. to twilights particularly now that baffles have been installed GM - produce a FAQ or "For the impatient" page addressing clearly how CFG to derive magnitudes, errors and coordinates etc. STH EGS EGS - continue to investigate how the seeing=-1 QC check information is used as part of our pre-release checks and see why/if this was missed MJI - release October WFCAM data New actions ----------- EGS - send WFAU a text file list with the latest VST/VISTA files and grades SC - update system software on apm48 and see what breaks GM - take over VISTA raw data tape backups and bring them up to date MJI - finish off calibration and release November and December WFCAM data EGS - update the VISTA long term ZP trend plots MJI - finish off "fix"es and release VST data for October -> December incl. JRL - test out supplmentary astrometry fields for HAWKI and VIMOS