Dark Exposure Properties of VIRCAM Detectors

Document number VDF-TRE-IOA-00008-0012 (Draft 20061021)


Jim Lewis and Sam Newman

1. Introduction

In this document we are looking at the dark properties of some of the AIT data that was taken at RAL with the 16 VIRCAM detectors. In what follows we use data that were taken with a variety of exposure times and numbers of DITs. Below are example dark frames for each of the VIRCAM detectors. These are combined 5s dark frames with NDIT = 1. The original images are VIRCAM_IMG_DARK255_000[1-5].fits.
figure 1 detector 1
Figure 1 Detector 1
 figure 2 detector 2
Figure 2 Detector 2
 figure 3 detector 3
Figure 3 Detector 3
 figure 4 detector 4
Figure 4 Detector 4
figure 5 detector 5
Figure 5 Detector 5
 figure 6 detector 6
Figure 6 Detector 6
 figure 7 detector 7
Figure 7 Detector 7
 figure 8 detector 8
Figure 8 Detector 8
figure 9 detector 9
Figure 9 Detector 9
 figure 10 detector 10
Figure 10 Detector 10
 figure 11 detector 11
Figure 11 Detector 11
 figure 12 detector 12
Figure 12 Detector 12
figure 13 detector 13
Figure 13 Detector 13
 figure 14 detector 14
Figure 14 Detector 14
 figure 15 detector 15
Figure 15 Detector 15
 figure 16 detector 16
Figure 16 Detector 16


There are several pretty obvious features we need to discuss:

2. Hot Pixels

Most of the detectors are peppered with hot pixels. If we take a closer look at a region where there are some hot pixels we see some interesting features. Figure 17 is a closeup of a small region of detector 1 centred roughly on the pixel [1920,1970].

figure 17 part of detector 1
Figure 17. A seection of detector 1 showing bad pixels before subutracton by mean sky.

This is the combined file from the previous section so none of these are likely to be cosmic rays. One thing that is noticeable straight away is that many of the bright pixels are part of a criss-cross pattern. Those circled in green have been marked for reference. For this criss-cross pattern the central pixel has a much higher value than the others, leading to the hypotheis that the central pixel is in fact a hot pixel and that the other four are good pixels that have been  affected by an interpixel correlation similar to what is seen in WFCAM. If this is the case, then the adjacent pixels should correct out with a simple dark subtraction. The table below shows a cut along a row through one of these hot pixel crosses for five different dark frames (region [1896:1905,1934:1934] in VIRCAM_IMG_DARK255_000[1-5].fits. The bright central pixel does indeed vary by rather a lot and a simple subtraction of a mean frame will certainly leave a significant positive or negative residual. The variation in the flux in the pixels on either side, however seem not to vary a great deal, indicating that subtracting a mean dark would probably correct them.

Table 1. Horizontal cuts through a cross on 5 different frames
-3
0
 -3
 4
 36
 19645
 123
 14
 3
 1
2
 -4
 3
 -6
 32
 19698
 131
 7
 -5
 0
3
 3
 4
 -1
 34
 19368
 121
 8
 -1
 9
-1
 8
 -9
 3
 37
 19440
 121
 7
 12
 2
9
 -5
 12
 0
 41
 19511
 124
 16
 1
 3

Figures 18 and 19 below show two dark frames that have been corrected by subtracting a mean dark. The green circle are on the same features as in figure 17.
Figure 18 dark corrected dark frame
Figure 18 A dark corrected dark frame
 Figure 19 Another dark corrected dark frame
Figure 19 Another dark corrected dark frame

These two figures show that the outside pixels in the cross pattern do indeed subtract out leaving either a positive or negative residual for the central pixel.  From this we can form an estimate of the number of hot pixels on each detector. We subtract the mean dark from all five of the original input dark frames. Then we search for either positive or negative 5 sigma points. Any pixel that appears as a 5 sigma point on 2 or more frames is flagged as a hot pixel. Using this method we have the following estimate for hot pixels in each detector.
Table 2. The percentage of hot pixels for each detector
Detector
 % hot pixels
1
 0.64
2
 0.04
3
 0.43
4
 0.27
5
 0.05
6
 0.21
7
 0.21
8
 1.56
9
 0.18
10
 0.02
11
 0.23
12
 0.42
13
 0.79
14
 1.12
15
 0.13
16
 1.13

Figures 20 and 21 below are the same as Figures 18 and 19, except that the bad pixels found by the method described above have been interpolated out. The fact that most of the bright pixels that are left are not common to the two images shows that we have eliminated most of the hot pixels and are left only with the transient features, i.e. cosmic rays.
Figure 20 dark corrected dark frame
Figure 20 A dark corrected dark frame with bad pixels masked out
 Figure 21 Another dark corrected dark frame
Figure 21 Another dark corrected dark frame with the bad pixels masked out

To do a dark correction properly it is reasonable to expect that you must have a master dark frame whose exposure time matches your target observation, simply from the point of subtracting out the correct amount of dark current. In an emergency situation does arise where no matching master dark is available, then it would be helpful to know whether the pixels adjacent to the hot pixels can still be corrected by subtracting a master dark with a mismatched exposure time. Figure 22 below shows the results of correcting a 4s dark exposure with a 2s master dark. Comparing this to figures 18 and 19 definitely shows that in order to correct the adjacent pixels, we must use a master dark with a matching exposure time.
figure 22 A 4s dark corrected by a 2s master dark
Figure 22 A 4s dark corrected by a 2s dark

VIRCAM will also produce images that are coadded at the telescope and logical also dictates that such images must be dark corrected using coadded dark frames. (That is we need to match the exposure time and the number of DITs between the target observation and the master dark). Logic also says that we should be able to correct a coadded image by scaling it and the master dark frame to NDIT=1. This would certainly remove the dark current, but what would be the effect on the hot pixels?  Figure 23 below shows the results of scaling an NDIT=2 5s dark frame by a half and then subtracting it from an NDIT=1 5s dark frame. The results here is comparable to what we got in Figures 18 and 19, which indicates that it is safe use master darks with mismatching values of NDIT, so long as the frames are all corrected to a value of NDIT=1 before processing.
A 5s NDIT=1 dark corrected by 1/2 * an NDIT=2 5s dark
Figure 23 A 5s NDIT=1 dark corrected by 1/2 * a 5s NDIT=2 dark frame

Before being replaced the original detector 10 had a huge glowing hot spot which didn't subtract very well. The new detector 10 doesn't have this sort of problem nor do any of the others. Detectors 14 and 15 do appear to have a hot pixel cluster each. Figure 24 below shows this for detector 15 on the mean dark frame. Figures 25 and 26 show two different dark images with the mean dark frame subtracted. The hot pixels in the cluster appear to over/under correct in the same way as the randomly distributed hot pixels in the images above and as such are probably no different from the latter. Figure 27 shows the same region as figure 26, but with the hot pixels that were defined above marked in red. This shows that these clusters can be flagged and treated in the same way as the randomly distributed hot pixels and that no special measure need to be invoked to take them into account.
Figure 24 Hot pixel cluster on detector 15 on mean dark frame
Figure 24 A hot pixel cluster on a mean dark frame for detector 15


figure 25
Figure 25 A region of a dark corrected dark frame of detector 15 showing the hot pixel cluster
 figure 26
Figure 26 A region of a different dark corrected dark frame of detector 15 showing the hot pixel cluster

figure 27
Figure 27 The same as figure 26, but with the pixels in the hot pixel mask marked in red

3. Curtains

The curtains that we saw in WFCAM data are also present in a slightly different form in the VIRCAM detectors. The quadrant structure in the WFCAM detectors is not present in the VIRCAM detectors, so that level of replication with rotation we had with WFCAM is missing in VIRCAM. However, figures 1-16 show that the same pattern repeats in groups of 4 detectors, corresponding to all the chips that are driven by the same controller. Having this level of redundancy is excellent news if we want to use the data themselves to correct the problem. The down side of this of course is that the pattern changes with every exposure, meaning that we can't do something like stacking a series of exposures throughout the night to get a mean 'curtain' frame. We can, however,
model the pattern by using data from all four detectors in a controller group for a particular exposure. We do this by forming the median background value for each row (ignoring object and bad pixels) in all four detectors. This gives a single 'stripe' 1d profile which can be normalised to zero median. The rows in the four detectors can then be corrected by subtracting the normalised value for that row from the stripe profile. Figure 28 below shows a dark exposure of detector 4 that has been corrected by subtracting a master dark. Figure 29 shows the same image that has been destriped with the algorithm described above. The hot pixel mask described in section 2 was used to mask out any remaining hot pixels from the groups of 4 detectors used to form the median profiles.

The normalised stripe patterns on average seem to  range between about +/- 15 counts. This is not a great deal, but is enough to add a certain amount of noise to the background of stacked images and as such it is worth removing it if we can.
figure 28. Dark corrected 5s dark frame (detector 4)
Figure 28: A dark corrected 5s dark frame (detector 4)
 figure 29 dark corrected 5s dark frame (detector 4) with stripes removed
Figure 29: A dark corrected 5s dark frame (detector 4) with stripes modelled and removed.

4. Dead Pixel Patches

Several of the chips have small patches of dead pixels, but none are as bad as detector 1. The dead pixel patches cover roughly 1.3% of the surface of detector 1 and are shown in figure 30 (the numbers in green are included for reference).
figure 30
Figure 30 A raw region of a raw dark image of detector 1 showing the dead pixel patches

The figures below (31-35) show the same region of five different dark images of detector 1 after subtracting a mean dark frame. What is interesting about the patches is how well patch 3 and most of patch 2 correct out by just doing a dark subtraction. In fact, the stripes from the curtains are visible in most of the patch areas in the corrected images which is interesting. Unfortunately when dark correcting images that have been exposed to light it becomes very clear that these regions don't flat field out and they are in fact a total loss.
figure 31
figure 32
figure 33#
figure 34
figure 35
Figures 31-35 The dead patch region of detector 1 in 5 dark corrected dark frames

5. Dark Current Maps

The table below shows the value of the dark current in average counts per second per pixel for each of the detector. This was done using the frames VIRCAM_IMG_DARK255_00138 to 153.  The hot pixels flagged in the earlier part of this document have been removed from the stats. The pixels that from the transepts of the cross patterns have not been removed. These do scale linearly with time which is why they subtract out properly in the first place. Unfortunately they make the dark current maps look much worse than they really should. The dark current maps for each detector is included in figures 36-51.

Detector
 Dark Current
[ADU/sec/pixel]
1
 0.45
2
 0.51
3
 0.93
4
 0.45
5
 0.32
6
 0.33
7
 0.38
8
 0.34
9
 0.35
10
 0.33
11
 0.35
12
 0.38
13
 0.94
14
 0.61
15
 0.32
16
 0.27

Figures 36-51. Dark current maps for all 16 VIRCAM detectors
figure 36
Figure 36 Detector 1
 figure 37
Figure 37 Detector 2
 figure 38
Figure 38 Detector 3 		figure 39
Figure 39. Detector 4
figure 40
Figure 40 Detector 5
 figure 41
Figure 41 Detector 6
 figure 42
Figure 42 Detector 7 		figure 43
Figure 43. Detector 8
figure 44
Figure 44 Detector 9
 figure 45
Figure 45 Detector 10
 figure 46
Figure 46 Detector 11 		figure 47
Figure 47. Detector 12
figure 48
Figure 48 Detector 13
 figure 49
Figure 49 Detector 14
 figure 50
Figure 50 Detector 15 		figure 51
Figure 51. Detector 16

6. Readnoise and Gain

The data that were taken as part of the AIT included flats that were meant to be part of the tests for linearity. VIRCAM_IMG_DARK255_0121,123 and VIRCAM_IMG_FLAT255_0022,24 are the four frames that were used to work out the readnoise and gain results presented below.
Detector
 Readnoise (e-)
 Gain (e-/ADU)
1
 22.0
 3.6
2
 21.4
 4.2
3
 21.2
 3.9
4
 24.3
 4.2
5
 22.7
 4.2
6
 20.4
 4.1
7
 22.8
 3.9
8
 29.3
 4.3
9
 21.7
 4.6
10
 24.5
 4.0
11
 27.5
 4.6
12
 25.8
 4.0
13
 30.8
 5.7
14
 26.5
 4.8
15
 19.9
 4.0
16
 24.0
 5.0

7. Duff Channels

Taking the difference of two darks (VIRCAM_IMG_DARK255_0103,105) shows a channel on detector 14 that is inconsistent with the rest of the channels in the detector. This is the 12th channel in the detector and needs to be addressed by RAL.
figure 52
Figure 52 A difference of two dark frames on detector 14 showing a channel which is inconsistent with the rest.

8. Some Conclusions About Dark Correction