"bad_pixel_offset_sigma = 5. # Any pixel whose offset is beyond 5 standard deviations, is a bad pixel\n",
"bad_pixel_noise_sigma = 5. # Any pixel whose noise is beyond 5 standard deviations, is a bad pixel\n",
"sigmaNoise = 5. # Any pixel whose signal exceeds 'sigmaNoise'*noiseCM (common mode corrected noise) will be masked\n",
"fix_temperature = False # This is required when the temperature is fixed in calibration database\n",
"fix_temperature = 0. # This is required when the temperature is fixed in calibration database\n",
"temperature_k = 233 # This is only used in the case of a fixed temperature for the calibration database\n",
"chunkSize = 100 # Number of images to read per chunk\n",
"cpuCores = 40 # Specifies the number of running cpu cores\n",
...
...
@@ -219,15 +219,17 @@
"\n",
"print(\"Bias voltage is {} V\".format(bias_voltage))\n",
"print(\"Detector gain is set to x{}\".format(det_gain), \"({} gain)\".format(gain_setting))\n",
"print(\"Detector integration time is set to {}\".format(integration_time), 'ms') \n",
" \n",
"if fix_temperature:\n",
"print(\"Detector integration time is set to {}\".format(integration_time), 'ms')\n",
"\n",
"temperature_k = fix_temperature + 273.15\n",
"if fix_temperature != 0.:\n",
" print(\"Using a fixed temperature of {}\".format(temperature_k), \"K\")\n",
" print(\"Mean temperature was {:0.2f} °C / {:0.2f} K\".format(temperature, temperature + 273.15))\n",
"else:\n",
" # This is needed while sending the \n",
" # calibration constant to the DB later\n",
" temperature_k = temperature + 273.15\n",
" print(\"Temperature is not fixed.\")\n",
"print(\"Mean temperature was {:0.2f} °C / {:0.2f} K\".format(temperature, temperature_k))\n",
"print(\"Mean temperature was {:0.2f} °C / {:0.2f} K\".format(temperature, temperature + 273.15))\n",
"\n",
"print(\"Output: {}\".format(out_folder))"
]
...
...
%% Cell type:markdown id: tags:
# FastCCD Dark Characterization
Author: I. Klačková, S. Hauf, K. Setoodehnia and M. Cascella
The following notebook provides dark image analysis of the FastCCD detector.
Dark characterization evaluates offset and noise of the FastCCD detector, corrects the noise for Common Mode (CM), and defines bad pixels relative to offset and CM corrected noise. Bad pixels are then excluded and CM corrected noise is recalculated excluding the bad pixels. Resulting offset and CM corrected noise maps, as well as the bad pixel map are sent to the calibration database.
Characterization of dark images with purpose to create dark maps (offset, noise and bad pixel maps) is an iterative process. Firstly, initial offset and noise maps are produced from raw dark data.
During the second iteration, the data are offset corrected and then common mode corrected to produced a common mode corrected noise map. The common mode correction is calculated by subtracting out the median of all pixels that are read out at the same time along a row.
In the following, the effect of common mode correction on the noise is shown. Finally common mode corrected noise map (noiseMapCM) is displayed and compared to the initial uncorrected noise map:
%% Cell type:code id: tags:
``` python
#*****NOISE MAP HISTOGRAM FROM THE COMMON MODE CORRECTED DATA*******#
hn,cn=np.histogram(noiseMap.flatten(),bins=200,range=(2,40))# hn: histogram of noise, cn: bin centers for noise
During the third iteration, the bad pixel map is applied to the data. Bad pixels are masked. Offset and common mode corrections are applied once again to the data, which now have bad pixdels excluded, to produce a common mode corrected noise map:
%% Cell type:code id: tags:
``` python
# bad_pixels is an array of (1934, 960, 1) filled with zeros except at indices where we have the actual bad pixels, whose
# values are set to be: 2 (2^1: BadPixels.OFFSET_OUT_OF_THRESHOLD.value), or
# 262144 (2^18: BadPixels.OVERSCAN.value), or 524288 (2^19: BadPixels.NON_SENSITIVE.value). These indices can be found
# using np.argwhere(bad_pixels != 0)
event_threshold=sigmaNoise*np.median(noiseMapCM)# for exclusion of possible cosmic ray events
The effect of exclusion of bad pixels on common mode corrected noise is shown below. Finally common mode corrected noise map with bad pixels excluded (noiseMapCM_2nd) is displayed:
%% Cell type:code id: tags:
``` python
#*****NOISE MAP HISTOGRAM FROM THE COMMON MODE CORRECTED DATA*******#
Lastly, the final bad pixel map is generated based on the OffsetMap and the noiseMapCM_2nd (common mode corrected noise after exclusion of the initial bad pixels):
print("Number of bad pixels: {:0.0f}, i.e. {:0.2f}% of all pixels".format(num_bad_pixels,percentage_bad_pixels))
```
%% Cell type:markdown id: tags:
### Electronic Noise
According to Table 6.1 (page 80) of Ivana Klačková's master's thesis: "Conversion gain for the FastCCD is: lower hemisphere = 6.2e-/ADU and upper hemisphere = 6.1e-/ADU."
The following Tables present the noise along lower hemisphere, upper hemisphere, and the entire FastCCD detector at different stages. Here, the values in the first table (in ADU and e-) are the mean of noise per pixel, where noise is considered to be the initial uncorrected noise, CM corrected noise after second trial (including bad pixels) and CM corrected noise after third trial (excluding bad pixels).
The values of the second table (in electrons) are the standard deviation of noise per pixel.
%% Cell type:code id: tags:
``` python
# noiseMap refers to the initial uncorrected noise, noiseMapCM refers to common mode corrected noise with inclusion of
# bad pixels, and noiseMapCM_2nd refers to common mode corrected noise without inclusion of bad pixels:
ADU_to_electron=(ADU_to_electron_upper+ADU_to_electron_lower)/2# Average of ADU_to_electron for the entire detector
