"commonModeAxis = 'row' # axis along which common mode will be calculated (0: along rows, 1: along columns)\n",
...
...
%% Cell type:raw id: tags:
%% Cell type:markdown id: tags:
# pnCCD Dark Characterization
Author: DET Group, modified by Kiana Setoodehnia, Version 2.0
The following notebook provides dark image analysis of the pnCCD detector. Dark characterization evaluates offset and noise of the detector and gives information about bad pixels.
On the first iteration, the offset and noise maps are generated. Initial bad pixels map is obtained based on the offset and initial noise maps.
On the second iteration, the noise map is corrected for common mode. A second bad pixel map is generated based on the offset map and offset-and-common-mode-corrected noise map. Then, the hole in the center of the CCD is added to the second bad pixel map.
On the third and final iteration, the pixels that show up on the abovementioned bad pixels map are masked. Possible events due to cosmic rays are found and masked. The data are then again offset and common mode corrected and a new final noise and bad pixels maps are generated.
These latter resulting maps together with the offset map are saved as .h5 files to a local path for a later use. These dark constants are not automatically sent to the database.
Characterization of dark images with purpose to create dark maps (offset, noise and bad pixel maps) is an iterative process. The initial offset and noise maps are produced from the raw dark data.
%% Cell type:code id: tags:
``` python
counter1=1# To count how many "if data.shape[2] >= chunkSize" instances are there.
counter2=0# To count how many "if data.shape[2] < chunkSize" instances are there.
chunkSize_new=0# See below
fordatainreader.readChunks():
data=data.astype(np.float32)
dx=np.count_nonzero(data,axis=(0,1))
data=data[:,:,dx!=0]
# Some sequences may have less than 500 frames in them. To find out how many images there are, we will
# temporarily change chunkSize to be the same as whatever number of frames the last chunk of data has:
ifdata.shape[2]<chunkSize:
chunkSize_new=data.shape[2]
print("Number of images are less than chunkSize. chunkSize is temporarily changed to {}."
.format(chunkSize_new))
images=images+chunkSize_new
counter2+=1
else:
images=counter1*chunkSize+counter2*chunkSize_new
counter1+=1
noiseCal.fill(data)# Filling the histogram calculator with data
chunkSize=100# resetting the chunkSize to its default value for the next sequence or data-chunk
print('A total number of {} images are processed.'.format(images))
### Offset and Noise Maps prior to Common Mode Correction
In the following, the histograms of the pnCCD offset and initial noise, and the heat maps of pnCCD offset, as well as the initial uncorrected noise are plotted.
%% Cell type:code id: tags:
``` python
#************** OFFSET MAP HISTOGRAM ***********#
ho,co=np.histogram(offsetMap.flatten(),bins=2000)# ho = offset histogram; co = offset bin centers
do={'x':co[:-1],
'y':ho,
#'y_err': np.sqrt(ho[:]), Markus thinks these errors bars are not correctly calculated!
During the second iteration, the data are offset and common mode corrected to produce an offset-and-common-mode-corrected noise map. The common mode correction is calculated by subtracting out the median of all pixels that are read out at the same time along a row.
%% Cell type:code id: tags:
``` python
counter1=1# To count how many "if data.shape[2] >= chunkSize" instances are there.
counter2=0# To count how many "if data.shape[2] < chunkSize" instances are there.
chunkSize_new=0# See below
fordatainreader.readChunks():
data=data.astype(np.float32)
dx=np.count_nonzero(data,axis=(0,1))
data=data[:,:,dx!=0]
# Some sequences may have less than 500 frames in them. To find out how many images there are, we will
# temporarily change chunkSize to be the same as whatever number of frames the last chunk of data has:
ifdata.shape[2]<chunkSize:
chunkSize_new=data.shape[2]
print("Number of images are less than chunkSize. chunkSize is temporarily changed to {}."
.format(chunkSize_new))
images=images+chunkSize_new
counter2+=1
else:
images=counter1*chunkSize+counter2*chunkSize_new
counter1+=1
data-=offsetMap.data# Offset correction
offset_corr_data=copy.copy(data)# I am copying this so that I can have access to it in the table below
histCalCorrected.fill(data)
cellTable=np.zeros(data.shape[2],np.int32)# Common mode correction
In the following, the effect of common mode correction on the noise is shown. Finally common mode corrected noise map (noiseMapCM) is displayed and compared to the initial uncorrected noise map.
%% Cell type:code id: tags:
``` python
#***** NOISE MAP HISTOGRAM FROM THE COMMON MODE CORRECTED DATA *******#
y_range=(0,x),vmin=0,vmax=32,panel_x_label='Row Stat (ADU)',
panel_y_label='Column Stat (ADU)',title='Second Bad Pixels Map with the Hole in Center')
```
%% Cell type:markdown id: tags:
### Third Iteration
During the third iteration, the last bad pixel map is applied to the data. Bad pixels are masked. Possible cosmic ray events are also masked. Offset and common mode corrections are applied once again to the data, which now have bad pixdels excluded, to produce a newly corrected noise map.
%% Cell type:code id: tags:
``` python
event_threshold=sigmaNoise*np.median(noiseMapCM)# for exclusion of possible cosmic ray events
noiseCal.setBadPixelMask(bad_pixels!=0)# setting bad pixels map for the noise calculator
```
%% Cell type:code id: tags:
``` python
counter1=1# To count how many "if data.shape[2] >= chunkSize" instances are there.
counter2=0# To count how many "if data.shape[2] < chunkSize" instances are there.
chunkSize_new=0# See below
fordatainreader.readChunks():
data=data.astype(np.float32)
dx=np.count_nonzero(data,axis=(0,1))
data=data[:,:,dx!=0]
data_mask=np.repeat(bad_pixels,data.shape[2],axis=2)# Converting bad_pixels to the same shape as the data
data[data_mask!=0]=np.nan# masking data for bad pixels and equating the values to np.nan
# Some sequences may have less than 500 frames in them. To find out how many images there are, we will
# temporarily change chunkSize to be the same as whatever number of frames the last chunk of data has:
ifdata.shape[2]<chunkSize:
chunkSize_new=data.shape[2]
print("Number of images are less than chunkSize. chunkSize is temporarily changed to {}."
title='Final Common Mode Corrected Noise\n (Bad Pixels Excluded)',
panel_x_label='Row Stat (ADU)',panel_y_label='Column Stat (ADU)')
```
%% Cell type:markdown id: tags:
### Final Bad Pixel Map
Lastly, the final bad pixel map is generated based on the OffsetMap and the noiseMapCM_2nd (offset and common mode corrected noise after exclusion of the bad pixels).
