CASU meeting Wednesday 27th June 11:30-13:00 back in APM Present: MJI, JRL, AKY, SC, JRF, EGS, JPE Apologies: NAW, MR, STH, RGM Agenda: 1. Actions from last meeting 2. Comments on WFAU minutes 3. Hardware update 4. Meetings and telecons 5. Data archives update 6. WFCAM update 7. VISTA update 8. VST update 9. Spectroscopic stuff 10. ESO in-kind stuff 11. AOB/news 1. Actions from last meeting ----------------------------- JRL - has actually finally gone and done it and investigated "rolling" master linearity corrections - see VISTA update section STH - is as ever consistent with progress of VISTA calibration papers, we can but hope for draft versions in time for next meeting <<<< JPE - have not yet characterised radial extent of diffraction spikes for a JRF range of values of NDIT*DIT for pawprints, or for tiles <<<< SC - reports a new G-bit switch was ordered and it may even have arrived !! EGS - will indeed be using apm49 as a (temporary ?) server and meanwhile tuned postgres system params for apm25 SC - home disk space has improved somewhat with assorted ancient accounts MJI being animatedly suspended EGS - are still rationalising serving up of UVEX data after the U-band <<<< MJI recalibration exercise MJI - did the avconf columns updates for all pre-20111001 WFCAM data SC - will continue investigation of computer montoring web page foibles when the new switch is installed <<<< MJI - regenerated the UVEX band-merged products and served them up MJI - did May 2012 WFCAM zero-point updates JRL - checked for lurking sky frames but the relevant one ne marche pas MJI - is still keeping an eye out for UCAC4 <<<< EGS - made sure we got permission to copy Chilean u-band ATLAS region data AKY and did, and processed it too MJI - produced quarterly report for ESO in-kind deliverables JRL 2. WFAU minutes --------------- Still piles of transfer speed issues via UKLight and equally mystifying obscurity regarding the cause of the bottleneck. SC will contact network support downtown and see if they can shed some light since all internal tests here reveal nothing wrong. <<<< The holes in the sky in WFAU-generated VMC deep Ks-band stacks were tracked down to an interesting use of scale factors in compressed confidence maps. 3. Hardware update ------------------- A few more issues regarding subtly different system software configurations on apm45-49 came to light. SC is busy tracking them down and fixing them. The exertion of STH's transient alert machine hopping onto the table clearly took its toll and it is still recovering, and oddly enough still awaiting its Infiniband card. All the recent wfcam data was re-arranged to safeguard against repeat problems on apm24_f, which as noted before, now hosts raw wfcam data only. JRF would like to be added to the VIRCAM email exploder so he can read at great length the nightly reports. <<<< 4. Meetings and telecons ------------------------- The UKIDSS final workshop was somewhat overshadowed the by the timely STFC announcement to discontinue support for UKIRT from the end of September 2013. This did at least stimulate lots of discussion on possible ways to extend access while JCMT still continues to operate. JRL (+Sergey) hauled over to Keele for a discussion of spectral processing foibles with Rob Jeffries and co. EGS tuned into the June 12th VMC skypecon and reported on processing progress and the strange saga of the sticking March nights. JPE, JRL and MJI were held spellbound by the latest VISTA IOT on June 22nd and had their minds erased while they were under. 5. Data archives update - AAT, ING, WFCAM, UKIRT, VISTA ------------------------ The archives are all up to date. On the processing front, much INT WFC for the latest Kepler runs and some more ancient data massaging for UVEX to propagate the U-band calibration update to the band-merged products and reserve the lot through one portal. 6. WFCAM update ---------------- May processed is done and dusted but not released since it requires MR to DB ingest it for its final checks. This will happen as soon as he gets back from vacation. <<<< After the standdown break, raw data acquistion, ingest and processing has resumed as normal A full set of skyflats including the latest 1.644FeII narrow band filter was obtained a toute vitesse together with even more emails discussing the UWISH2 and FeII observing strategy ......result sky flipping encore plusieurs fois !! There were, d'accord, the usual few minor foibles to workaround with the advent of a new filter (PV2_3 .... etc..). These were quickly spotted and sorted, so much so that we already have a satisfied FeII customer. As noted earlier, during the standdown the threatened updates to the avconf column for 20050401-20110930 incl. were made and the much misaligned version numbering rearranged to coincide with WFAU's, so -> v5 it is then. 7. VISTA update ---------------- Receipt of data, ingest, checking and processing going well apart from a couple of recalcitrant March nights which are proving hard to shift. Rather than waiting for eternity to begin, the rest of the March data and all of the April data were released. At the time of writing that just leaves 20120330 to kick out of the door - jammed apparently due to a hole in the 2MASS coverage around a particularly bright star (Sol). The rest of the foibles could be traced to incomplete data transfers for a few files. MJI assumed (d'er!) that a direct check on the number of extensions present was done, but ..... it wasn't ..... and ...... now is. MJI noted a comment from Valentin about spikier than usual darks possibly caused by IRAC controller(s). JRL will check if the darks dqc info picks this up and see if the spikes are actually getting worse. <<<< Matt Jarvis sent around a VIDEO survey paper for comments, of which there were plenty, though quite how many made it into the submitted version is a moot point. JPE is doing his usual grand job on calibration issues which raises the question of where we have got on the offset between VISTA Y and Z and Vega. Recall for VISTA we decided to publish these explicitly as offsets to be applied rather than footling around with the Zpts, i.e. staying with the the VISTA system mantra and leaving it to users to convert to whatever other system they want. These offsets and the details of their derivation should be appearing in the calibration paper(s). STH will report on progress next meeting. <<<< To add to the ongoing VVV data saga: apparently VVV Chile can't read 3 of the LTO4 tapes we sent despite earlier claiming said tapes were ok. They think they will be able to copy the missing files over the Internet - we will no doubt see. Eammon Kerins is being strongly encouraged to grab any further VVV data monthly via Internet transfers. There was some discussion regarding ESO Phase3 deliveries. As far as we can tell ESO do not want any further images etc.. until July 1st and only required band-merged catalogues until then. After July 1st there are two months to make the next phase of deliveries of images, confidence maps, usw.. So far VVV and VHS (and possibly VMC) have signed up CASU for this next act. JPE volunteered to see what VIKING were doing about image et al. deliveries. <<<< EGS has been fending off requests from Terrapix to deliver all VIKING data by carrier pigeon and instead suggested they try Internet transfers. It appears they want to run the deep stack sky background masking method on it, which is somewhat puzzling as a.) it isn't and b.) it won't make any difference. VISTA PI data for Thorsten Ratzka 088.C-0117(A) was signed sealed and delivered, the odd March night notwithstanding. JPE asked about PSF fitting software plans which was timely as a similar request came via some recent WFAU minutes. To recap:- The PSF fitting software add on to the current catalogues exists, i.e. to generate PSFs automatically and apply them at pre-determined locations, but as noted by our friendly GPS and VVV folk does not address the real problem with the current catalogues in very crowded regions. To sort this properly needs a DAOPHOT-like solution to regenerate a more complete list of objects first and then PSF'em. Hard to give a timescale for this but we plan to work on it over the summer and test the feasibility of doing this completely automatically and releasing it as an optional CASUtools piece of kit. Further updates will be reported in our minutes. As promised earlier, herewith JRL's assessment of the linearity data: -- there is some slow evolution of the linearity. Data from June 2010 doesn't linearise as well with a May 2012 linearity table as it does with the June 2010 table. You can definitely see some curvature, but that said the residuals are small compared to the fluxes (e.g. a few tenths of a percent at the mid and bright ends, but about 1-2% at the faint end). -- the detector channels do need to have independent linearity curves. Attempting to linearise the detectors with a mean curve gives visibly worse results. -- using linearity data from January, April and May 2012 and June 2011 to form a mean linearity curve (e.g. take 4 separate linearity curves and then averaged the coefficients for each detector channel). For most of the detectors the averaged linearity corrected data seems if anything to be a bit better than with the single linearity curve. The exception to this appears to be detectors 9-12 which seem to vary a bit more with time than the others. Again, the scale of the variation appears to be small. Channel variations disappear between runs using this method. -- saturation doesn't appear to be varying much, if at all. But it's hard to tell because of the way ESO takes linearity sequences -- suggestion is that we find a couple of nights that and use as a test for a mean linearity curve; reduce them and see if we get any major differences between the reduction with the mean curve and a reduction done with the current curve. <<<< PS. We haven't forgotten to progress apcor2 fixes at some point. VST --- VST data receipt, ingest and processing is proceeding well. The Chilean u-band OBs are being grabbed and processed with the rest, after an initial catchup phase. For now no deeper stacking of u-band data for ATLAS is planned since the next phase will use list-driven (from g-band) data to derive catalogues, which can be "stacked" as needed. The first official release aka v1.0 up to the end of May 2012 happened and data released to the PIs. This release includes the illumination correction fixed versions of the catalogues but is still based on the earlier zero-points. The planned updates of the zero-points awaits the next version of APASS which should give much fuller sky coverage. As noted previously APASS will be used to derive improved nightly zero-points and also an alternative AB-mag set. This reminded MJI to ask JRL to check for new APASS release since rumour has it that v6 is out. <<<< The released fixed ATLAS catalogues were checked independently by Nigel Metcalfe and passed with flying colours, but yet again raised the main remaining issue, the random gain change foible in some of the VST detectors. This awaits an updated bandmerging prog for VPHAS data before being tried in anger. <<<< 9. Spectroscopic stuff ----------------------- GaiaESO pipeline tests e.g. derived velocities, are looking good and the release DB progess is sufficient to keep (most) users happy. EGS continues to analyse simulated spectra for 4MOST and is succeeding in shoehorning requirements, FOMs, and science goals into a coherent story. NAW and MJI chipped in at the telecon level to rationalise the amount of work requested, by mainly arguing to drop the muddled round 3 exercise given the cockus upus status of the simulated spectra. By refocussing on round 4 (VISTA only) and by pushing for an ETC to help PIs set sensible exposure times we expect to streamline the ongoing process. 10. ESO in-kind stuff --------------------- As noted in the completed actions MJI wrote the quarterly report and JRL wrote a detailed design doc for the VISTA pipeline upgrades, though we still haven't reveived a copy of the as-signed SOW. HAWKIng proceeds apace with the installation of the latest Reflex workflow with Kepler actors (whatever they are). 11. AOB/news ------------ Not a lot but xxxx gets the prize for the most amusing CASUhelp of the month. Continuing actions ------------------ STH - progress evermore the VISTA calibration papers writeups and circulate draft papers before the next CASU meeting JPE - continue to characterise radial extent of diffraction spikes for a JRF range of values of NDIT*DIT for pawprints, and thence tiles EGS - rationalise serving up of UVEX data MJI SC - continue investigation of computer montoring web page foibles when the new switch is installed MJI - is still keeping an eye out for UCAC4 New Actions ----------- SC - contact UCS network support about UKLight network speed issues SC - add JRF to VIRCAM email exploder MR - do DB ingest for May WFCAM data prior to releasing it JRL - check for spikier than usual darks STH - report on progress on VISTA Y and Z Vega offsets JPE - inquire about VIKING Phase3 deliveries of images et al. JRL - check for new APASS release (v6) and grab it if so MJI - update bandmerging prog to cope with weird VPHAS observing pattern