"ADU_to_electron_lower_lg = 47.3 # for lower hemisphere and low gain\n"
]
},
{
...
...
%% Cell type:markdown id: tags:
# FastCCD Dark Characterization
Author: I. Klačková, S. Hauf, K. Setoodehnia and M. Cascella
The following notebook provides dark image analysis of the FastCCD detector.
Dark characterization evaluates offset and noise of the FastCCD detector, corrects the noise for Common Mode (CM), and defines bad pixels relative to offset and CM corrected noise. Bad pixels are then excluded and CM corrected noise is recalculated excluding the bad pixels. Resulting offset and CM corrected noise maps, as well as the bad pixel map are sent to the calibration database.
Characterization of dark images with purpose to create dark maps (offset, noise and bad pixel maps) is an iterative process. Firstly, initial offset and noise maps are produced from raw dark data.
%% Cell type:code id: tags:
``` python
counter1=1# To count how many "if data.shape[2] >= chunkSize" instances are there.
counter2=0# To count how many "if data.shape[2] < chunkSize" instances are there.
During the second iteration, the data are offset corrected and then common mode corrected to produced a common mode corrected noise map. The common mode correction is calculated by subtracting out the median of all pixels that are read out at the same time along a row.
%% Cell type:code id: tags:
``` python
counter1=1# To count how many "if data.shape[2] >= chunkSize" instances are there.
counter2=0# To count how many "if data.shape[2] < chunkSize" instances are there.
In the following, the effect of common mode correction on the noise is shown. Finally common mode corrected noise map (noiseMapCM) is displayed and compared to the initial uncorrected noise map:
%% Cell type:code id: tags:
``` python
#*****NOISE MAP HISTOGRAM FROM THE COMMON MODE CORRECTED DATA*******#
hn,cn=np.histogram(noiseMap.flatten(),bins=200,range=(0,40))# hn: histogram of noise, cn: bin centers for noise
During the third iteration, the bad pixel map is applied to the data. Bad pixels are masked. Offset and common mode corrections are applied once again to the data, which now have bad pixdels excluded, to produce a common mode corrected noise map:
%% Cell type:code id: tags:
``` python
# bad_pixels is an array of (1934, 960, 1) filled with zeros except at indices where we have the actual bad pixels, whose
# values are set to be: 2 (2^1: BadPixels.OFFSET_OUT_OF_THRESHOLD.value), or
# 262144 (2^18: BadPixels.OVERSCAN.value), or 524288 (2^19: BadPixels.NON_SENSITIVE.value). These indices can be found
# using np.argwhere(bad_pixels != 0)
event_threshold=sigmaNoise*np.median(noiseMapCM)# for exclusion of possible cosmic ray events
noiseCal.setBadPixelMask(bad_pixels!=0)
```
%% Cell type:code id: tags:
``` python
counter1=1# To count how many "if data.shape[2] >= chunkSize" instances are there.
counter2=0# To count how many "if data.shape[2] < chunkSize" instances are there.
The effect of exclusion of bad pixels on common mode corrected noise is shown below. Finally common mode corrected noise map with bad pixels excluded (noiseMapCM_2nd) is displayed:
%% Cell type:code id: tags:
``` python
#*****NOISE MAP HISTOGRAM FROM THE COMMON MODE CORRECTED DATA*******#
Lastly, the final bad pixel map is generated based on the OffsetMap and the noiseMapCM_2nd (common mode corrected noise after exclusion of the initial bad pixels):
print("Number of bad pixels: {:0.0f}, i.e. {:0.2f}% of all pixels".format(num_bad_pixels,percentage_bad_pixels))
```
%% Cell type:markdown id: tags:
### Electronic Noise
According to Table 6.1 (page 80) of Ivana Klačková's master's thesis: "Conversion gain for the FastCCD high gain is: lower hemisphere = 6.2e-/ADU and upper hemisphere = 6.1e-/ADU." Also, we know that the high gain/medium gain and high gain/low gain ratios are 4 and 8, respectively since high gain = x8, medium gain = x2 and low gain = x1. We do not currently (October - 2019) know the conversion gains for the FastCCD medium and lows gains in electrons. Therefore, we will use those of the high gains (in both hemispheres) together with the gain ratios to convert the noise in ADU to electrons.
The following Tables present the noise along lower hemisphere, upper hemisphere, and the entire FastCCD detector at different stages. Here, the values in the first table (in ADU and e-) are the mean of noise per pixel, where noise is considered to be the initial uncorrected noise, CM corrected noise after second trial (including bad pixels) and CM corrected noise after third trial (excluding bad pixels).
The values of the second table (in electrons) are the standard deviation of noise per pixel.
%% Cell type:code id: tags:
``` python
# noiseMap refers to the initial uncorrected noise, noiseMapCM refers to common mode corrected noise with inclusion of
# bad pixels, and noiseMapCM_2nd refers to common mode corrected noise without inclusion of bad pixels:
ADU_to_electron_hg=(ADU_to_electron_upper_hg+ADU_to_electron_lower_hg)/2# Average of ADU_to_electron for entire CCD
# for high gain
ADU_to_electron_mg=(ADU_to_electron_upper_mg+ADU_to_electron_lower_mg)/2# Average of ADU_to_electron for entire CCD
# for medium gain
ADU_to_electron_lg=(ADU_to_electron_upper_lg+ADU_to_electron_lower_lg)/2# Average of ADU_to_electron for entire CCD
# for low gain
```
%% Cell type:code id: tags:
``` python
forgain,valueingain_dict.items():
ifdet_gain==gain_dict["low gain"]:
ADU_to_electron=ADU_to_electron_lg
ADU_to_electron_upper=ADU_to_electron_upper_lg
ADU_to_electron_lower=ADU_to_electron_lower_lg
elifdet_gain==gain_dict["medium gain"]:
ADU_to_electron=ADU_to_electron_mg
ADU_to_electron_upper=ADU_to_electron_upper_mg
ADU_to_electron_lower=ADU_to_electron_lower_mg
else:# Here, we assume the auto gain and high gain conversions from ADU to electrons are the same.
