add gitlab links for notebooks and give warnings for unsupported det notebooks

5fcdbf76 · Karim Ahmed · 77673815 · 5fcdbf76
Commit 5fcdbf76 authored 1 year ago by Karim Ahmed
--- a/docs/operation/available_notebooks.md
+++ b/docs/operation/available_notebooks.md
@@ -5,18 +5,18 @@ The following notebooks are currently integrated into European XFEL Offline Cali

 ## AGIPD

-### AGIPD Correction
+### [AGIPD Correction](https://git.xfel.eu/calibration/pycalibration/-/blob/master/notebooks/AGIPD/AGIPD_Correct_and_Verify.ipynb)

 Offline Correction for AGIPD Detector


-### AGIPD Characterize Dark Images
+### [AGIPD Characterize Dark Images](https://git.xfel.eu/calibration/pycalibration/-/blob/master/notebooks/AGIPD/Characterize_AGIPD_Gain_Darks_NBC.ipynb)

 This notebook analyzes a set of dark images taken with AGIPD detector to deduce detector offsets (pedestal), noise, bad-pixel maps and thresholds. All four types of constants are evaluated per-pixel and per-memory cell. Data for the detector’s three gain stages needs to be present and separated into separate runs.

 The evaluated calibration constants are stored locally and injected in the calibration database.

-### Characterize AGIPD Pulse Capacitor Data
+### [Characterize AGIPD Pulse Capacitor Data](https://git.xfel.eu/calibration/pycalibration/-/blob/master/notebooks/AGIPD/Chracterize_AGIPD_Gain_PC_NBC.ipynb)


 The following code characterizes AGIPD gain via data take with the pulse capacitor source (PCS). The PCS allows scanning through the high and medium gains of AGIPD, by subsequently increasing the number of charge pulses from on-ASIC capacitor, thus increasing the charge a pixel sees
@@ -38,17 +38,17 @@ The same regions are present in the gain-bit data and are used to deduce the swi

 The resulting slopes are then fitted with a linear function and a combination of a linear and exponential decay function to determine the relative gains of the pixels with respect to the module. Additionally, we deduce masks for bad pixels form the data.

-### Gain Characterization
+### [Gain Characterization](https://git.xfel.eu/calibration/pycalibration/-/blob/master/notebooks/AGIPD/Characterize_AGIPD_Gain_FlatFields_NBC.ipynb)

 This notebook is used to produce AGIPD Flat-Field constants.

-### Histogramming of AGIPD FF data
+### [Histogramming of AGIPD FF data](https://git.xfel.eu/calibration/pycalibration/-/blob/master/notebooks/AGIPD/AGIPD_FF_Histogramming.ipynb)

 ??? warning "No description"

    TODO: Add description for this notebook

-### Combine Constants
+### [Combine Constants](https://git.xfel.eu/calibration/pycalibration/-/blob/master/notebooks/AGIPD/AGIPD_Characterize_Gain_Combine_NBC.ipynb)

 This notebook combines constants from various evaluations

@@ -67,12 +67,12 @@ Additionally, a bad pixel mask for all gain stages is deduced from the input. Th
 ## DSSC


-### DSSC Offline Correction
+### [DSSC Offline Correction](https://git.xfel.eu/calibration/pycalibration/-/blob/master/notebooks/DSSC/DSSC_Correct_and_Verify.ipynb)


 Offline Correction for DSSC Detector

-### DSSC Characterize Dark Images
+### [DSSC Characterize Dark Images](https://git.xfel.eu/calibration/pycalibration/-/blob/master/notebooks/DSSC/Characterize_DSSC_Darks_NBC.ipynb)

 The following code analyzes a set of dark images taken with the DSSC detector to deduce detector offsets and noise. Data for the detector is presented in one run and don’t acquire multiple gain stages.

@@ -82,7 +82,7 @@ The notebook explicitly does what `pyDetLib` provide in its offset calculation m

 ## EPIX100

-### ePix100 Data Correction
+### [ePix100 Data Correction](https://git.xfel.eu/calibration/pycalibration/-/blob/master/notebooks/ePix100/Correction_ePix100_NBC.ipynb)

 The following notebook provides data correction of images acquired with
 the ePix100 detector.
@@ -94,7 +94,7 @@ Offset, common mode and gain corrected data is saved to

 If pattern classification is applied (charge sharing correction), this data will be saved to /data/image/pixels_classified, while the corresponding patterns will be saved to /data/image/patterns in the CORR files.

-### ePix100 Dark Characterization
+### [ePix100 Dark Characterization](https://git.xfel.eu/calibration/pycalibration/-/blob/master/notebooks/ePix100/Characterize_Darks_ePix100_NBC.ipynb)

 The following notebook provides dark image analysis and calibration constants of the ePix100 detector.

@@ -115,7 +115,7 @@ Resulting maps are saved as HDF5 files for a later use and injected to calibrati

 ## GOTTHARD2

-### Gotthard2 Offline Correction
+### [Gotthard2 Offline Correction](https://git.xfel.eu/calibration/pycalibration/-/blob/master/notebooks/Gotthard2/Correction_Gotthard2_NBC.ipynb)

 Offline Correction for Gotthard2 Detector.

@@ -136,7 +136,7 @@ The correction is done per sequence file. If all selected sequence files have no
 The same result would be reached in case the needed dark calibration constants were not retrieved for all modules and `offset_correction` is True.
 In case one of the gain constants were not retrieved `gain_correction` is switched to False and gain correction is disabled.

-### Gotthard2 Dark Image Characterization
+### [Gotthard2 Dark Image Characterization](https://git.xfel.eu/calibration/pycalibration/-/blob/master/notebooks/Gotthard2/Characterize_Darks_Gotthard2_NBC.ipynb)

 The following is a processing for the dark constants (`Offset`, `Noise`, and `BadPixelsDark`) maps using dark images taken with Gotthard2 detector (GH2 50um or 25um).
 All constants are evaluated per strip, per pulse, and per memory cell. The maps are calculated for each gain stage that is acquired in 3 separate runs.
@@ -147,7 +147,7 @@ The three maps can be injected to the database and/or stored locally.

 ## JUNGFRAU

-### Jungfrau Offline Correction
+### [Jungfrau Offline Correction](https://git.xfel.eu/calibration/pycalibration/-/blob/master/notebooks/Jungfrau/Jungfrau_Gain_Correct_and_Verify_NBC.ipynb)

 Offline Correction for Jungfrau detector. For single and burst mode and for detector operated in adaptive or fixed gain.

@@ -166,7 +166,7 @@ The notebook accounts for the raw gain values (0[00], 1[01], 3[11]) during corre
    * The sensor instead has rectangular pixels: 25 um x 225 um, i.e. 1/3 of the pixel pitch along the x-axis and three times along the y-axis (to maintain the total number of pixel unchanged);
    * The readout however is not 'aware' of these changes, so it treats it like a 'normal' JUNGFRAU module, hence the output must be re-shuffled (in the row and column dimensions) in order for the image to make sense.

-### Jungfrau Dark Image Characterization
+### [Jungfrau Dark Image Characterization](https://git.xfel.eu/calibration/pycalibration/-/blob/master/notebooks/Jungfrau/Jungfrau_dark_analysis_all_gains_burst_mode_NBC.ipynb)

 Analyzes Jungfrau dark image data to deduce offset, noise and bad pixel maps from three dark runs of each gain. For data of single or burst mode and detector operated in adaptive or fixed gain.

@@ -174,6 +174,10 @@ Analyzes Jungfrau dark image data to deduce offset, noise and bad pixel maps fro

 `BadPixelsDark` calibration parameter consists of pixels with wrong gain values, empty cell images (cells with 0 pixel values), pixels for offset and noise maps evaluated with values above bad pixel threshold sigmas, and infinite values in offset or noise maps.

+### [Jungfrau Dark Summary](https://git.xfel.eu/calibration/pycalibration/-/blob/master/notebooks/Jungfrau/Jungfrau_darks_Summary_NBC.ipynb)
+
+Summary for process dark constants and a comparison with previously injected constants with the same conditions.
+
 ### References: 

 - [Jungfrau documentation](https://rtd.xfel.eu/docs/jungfrau-detector-documentation/en/latest/jf_correction.html)
@@ -182,7 +186,7 @@ Analyzes Jungfrau dark image data to deduce offset, noise and bad pixel maps fro

 ## LPD

-### LPD Offline Correction
+### [LPD Offline Correction](https://git.xfel.eu/calibration/pycalibration/-/blob/master/notebooks/LPD/LPD_Correct_Fast.ipynb)

 Offline correction for LPD detector data.

@@ -194,7 +198,7 @@ The notebook expects 6 different constants. `Offset`, `RelativeGain`, `FFMap`, `
 Offset –> RelativeGain * FFMap * GainAmpMap.


-### LPD Offset, Noise and Dead Pixels Characterization
+### [LPD Offset, Noise and Dead Pixels Characterization](https://git.xfel.eu/calibration/pycalibration/-/blob/master/notebooks/LPD/LPDChar_Darks_NBC.ipynb)

 This notebook process dark images to derive offset, noise and bad-pixel maps. All three types of constants are evaluated per-pixel and per-memory cell.

@@ -202,19 +206,19 @@ The notebook will correctly handle veto settings, but note that if you veto cell

 The evaluated calibration constants are stored locally and injected in the calibration database.

-### LPD Radial X-ray Gain Evaluation
+### [LPD Radial X-ray Gain Evaluation](https://git.xfel.eu/calibration/pycalibration/-/blob/master/notebooks/LPD/LPD_FlatField_Radial_per_pixel_CI_NBC.ipynb)

 Taking proper flat field for LPD can be difficult, as air scattering will always be present. Additionally, the large detector mandates a large distance to the source, in order to reduce $1/r$ effects.

 Because of this a radial evaluation method is used, which assumes that pixels are the same radial distance $r$ should on average have the
 same signal $S(r)$.

-#### Injecting calibration constant data to the database
+### [Injecting calibration constant data to the database](https://git.xfel.eu/calibration/pycalibration/-/blob/master/notebooks/LPD/Inject_calibration_constants_from_h5files.ipynb)

 Reading h5files of calibration constants to inject them to the database.
 Used for LPD

-### LPD Gain Characterization (Charge Injection)
+### [LPD Gain Characterization (Charge Injection)](https://git.xfel.eu/calibration/pycalibration/-/blob/master/notebooks/LPD/Characterize_LPD_GAIN_CI_per_pixel_NBC.ipynb)

 The following code characterizes the gain of the LPD detector from charge injection data, i.e. data with charge injected into the amplifiers, bypassing the sensor. The data needs to fulfil the following requirements:

@@ -232,7 +236,7 @@ The data is then analyzed by calculating the per-pixel, per memory cell mean of

 ## PNCCD

-### pnCCD Data Correction
+### [pnCCD Data Correction](https://git.xfel.eu/calibration/pycalibration/-/blob/master/notebooks/pnCCD/Correct_pnCCD_NBC.ipynb)

 The following notebook provides offset, common mode, relative gain, split events and pattern classification corrections of images acquired with the pnCCD. This notebook *does not* yet correct for charge transfer inefficiency.

@@ -241,7 +245,7 @@ Offset –> Common Mode –> Gain Correction –> Split pattern classification

 The notebook stores the offset corrected data at `/data/pixels`, the common mode corrected data at `/data/pixels_cm`, the gain constant used is stored at `/data/gain`, the pattern classification has two arrays stored `/data/pixels_classified` and `/data/patterns`, and the bad pixels are masked and stored at `data/mask`.

-### pnCCD Dark Characterization
+### [pnCCD Dark Characterization](https://git.xfel.eu/calibration/pycalibration/-/blob/master/notebooks/pnCCD/Characterize_pnCCD_Dark_NBC.ipynb)

 This notebook provides dark image analysis of the pnCCD detector. Dark characterization evaluates offset and noise of the detector and gives information about bad pixels.

@@ -253,7 +257,7 @@ On the third and final iteration, the pixels that show up on the above-mentioned

 These latter resulting maps together with the offset map are saved as HDF5 files to a local path for a later use. These dark constants are not automatically sent to the database.

-### pnCCD Gain Characterization
+### [pnCCD Gain Characterization](https://git.xfel.eu/calibration/pycalibration/-/blob/master/notebooks/pnCCD/Characterize_pnCCD_Gain.ipynb)

 The following notebook provides gain characterization for the pnCCD. It relies on EuXFEL offline corrected data. Prior to running this notebook, the expected applied corrections are:

@@ -269,7 +273,7 @@ The following notebook provides gain characterization for the pnCCD. It relies o

 ## REMI

-### Transformation parameters
+### [Transformation parameters](https://git.xfel.eu/calibration/pycalibration/-/blob/master/notebooks/REMI/REMI_Digitize_and_Transform.ipynb)

 ??? warning "No description"

@@ -279,11 +283,15 @@ The following notebook provides gain characterization for the pnCCD. It relies o

 ## EPIX10K

-### ePIX10K Data Correction
+??? warning "Detector notebooks are outdated."
+
+   EPIX10K notebooks are outdated as the detector hasn't been used for years.
+
+### [ePIX10K Data Correction](https://git.xfel.eu/calibration/pycalibration/-/blob/master/notebooks/ePix10K/Correction_ePix10K_NBC.ipynb)

 Offset correction of images acquired with the ePix10K detector.

-### ePix10K Dark Characterization
+### [ePix10K Dark Characterization](https://git.xfel.eu/calibration/pycalibration/-/blob/master/notebooks/ePix10K/Characterize_Darks_ePix10K_NBC.ipynb)

 Dark image analysis of the ePix10K detector.

@@ -293,11 +301,15 @@ Dark characterization evaluates offset and noise of the detector and gives infor

 ## FASTCCD

-### FastCCD Data Correction
+??? warning "Detector notebooks are outdated."
+
+   FastCCD notebooks are outdated as the detector hasn't been used for years.
+
+### [FastCCD Data Correction](https://git.xfel.eu/calibration/pycalibration/-/blob/master/notebooks/FastCCD/CorrectionNotebook_NewDAQ_FastCCD_NBC.ipynb)

 The following notebook provides correction of images acquired with the FastCCD.

-### FastCCD Dark Characterization
+### [FastCCD Dark Characterization](https://git.xfel.eu/calibration/pycalibration/-/blob/master/notebooks/FastCCD/Characterize_Darks_NewDAQ_FastCCD_NBC_New_Common_Mode.ipynb)


 The following notebook provides dark image analysis of the FastCCD detector.
@@ -309,16 +321,14 @@ detector, corrects the noise for Common Mode (CM), and defines bad pixels relati

 ## GENERIC

-### Constants from DB to HDF5
+### [Constants from DB to HDF5](https://git.xfel.eu/calibration/pycalibration/-/blob/master/notebooks/generic/DB_Constants_to_HDF5_NBC.ipynb)

 Currently, available instances are LPD1M1 and AGIPD1M1

-### Statistical analysis of calibration factors

-Plot calibration constants retrieved from the cal. DB.
+### [Overall modules darks summary](https://git.xfel.eu/calibration/pycalibration/-/blob/master/notebooks/generic/overallmodules_Darks_Summary_NBC.ipynb)

-To be visualized, calibration constants are averaged per group of pixels. Plot shows calibration constant over time for each constant.
-Values shown in plots are saved in h5 files.
+Create a summary chapter for the processed dark calibration constants and a comparison with previous injected constants. Only for AGIPDs, DSSC1M, and LPD1M

 ----------------------------------------------------------------------------------------