[LPD][DARK][CORRECT] Add LPD Mini support

Description

This MR modifies the LPD dark and correct notebooks to handle the LPD Mini at FXE.

I would particularly like feedback in the dark notebook, in which I copy-pasted functions from cal_tools.tools with a slight change, because it was easier than refactoring the file allocation in the limited time I have.
Ideally, it would become similar to what's in the correct notebook, but that's a major refactor for which I don't have the resources at the moment.
Perhaps you guys have a suggestion?

These rely on changes in extra-geom and extra-data, tags will need to be updated.

How Has This Been Tested?

Constants could be generated locally and injected to the database from some sample data, but were not verified for quality.
These constants worked well with CalNG and the correction notebook.

I've also injected fake GainAmpMap, RelativeGain, FFMap, BadPixelsFF constants, to test that the correction notebook works.

Relevant Documents (optional)

The dark notebook can be tested with the following parameters:

in_folder = "/gpfs/exfel/exp/FXE/202231/p900316/raw" # path to input data, required
out_folder = "/gpfs/exfel/data/scratch/danilevc/test/LPD/" # path to output to, required
metadata_folder = ""  # Directory containing calibration_metadata.yml when run by xfel-calibrate
sequence = 0 # sequence files to evaluate
modules = [-1] # list of modules to evaluate, RANGE ALLOWED
run_high = 10 # run number in which high gain data was recorded, required
run_med = 11 # run number in which medium gain data was recorded, required
run_low = 12 # run number in which low gain data was recorded, required

karabo_id = "FXE_DET_LPD_MINI" # karabo karabo_id
karabo_da = [-1]  # a list of data aggregators names, Default [-1] for selecting all data aggregators
receiver_id = "{}CH0" # inset for receiver devices
path_template = 'RAW-R{:04d}-{}-S{:05d}.h5' # the template to use to access data
h5path = '/INSTRUMENT/{}/DET/{}:xtdf/image' # path in the HDF5 file to images
h5path_idx = '/INDEX/{}/DET/{}:xtdf/image' # path in the HDF5 file to images

use_dir_creation_date = True # use the creation date of the directory for database time derivation
cal_db_interface = "tcp://max-exfl016:8015#8025" # the database interface to use
cal_db_timeout = 300000 # timeout on caldb requests"
local_output = True # output constants locally
db_output = False # output constants to database

capacitor_setting = 5 # capacitor_setting for which data was taken 
mem_cells = 512 # number of memory cells used
bias_voltage = 250 # detector bias voltage
thresholds_offset_sigma = 3. # bad pixel relative threshold in terms of n sigma offset
thresholds_offset_hard = [400, 1500] # bad pixel hard threshold
thresholds_noise_sigma = 7. # bad pixel relative threshold in terms of n sigma noise
thresholds_noise_hard = [1, 35] # bad pixel hard threshold
skip_first_ntrains = 10 # Number of first trains to skip

instrument = "FXE" # instrument name
ntrains = 100 # number of trains to use
high_res_badpix_3d = False # plot bad-pixel summary in high resolution
test_for_normality = False # permorm normality test
operation_mode = 'STANDARD_THREE_GAINS'  # Detector operation mode, optional

The Correct Fast notebook can be tested with the following parameters:

# Input parameters
in_folder = "/gpfs/exfel/exp/FXE/202231/p900316/raw"  # the folder to read data from, required
out_folder = "/gpfs/exfel/data/scratch/danilevc/test/LPD/"  # the folder to output to, required
metadata_folder = ''  # Directory containing calibration_metadata.yml when run by xfel-calibrate.
sequences = [-1]  # Sequences to correct, use [-1] for all
modules = [-1]  # Modules indices to correct, use [-1] for all, only used when karabo_da is empty
karabo_da = ['']  # Data aggregators names to correct, use [''] for all
run = 10  # run to process, required

# Source parameters
karabo_id = 'FXE_DET_LPD_MINI'  # Karabo domain for detector.
input_source = '{karabo_id}/DET/{module_index}CH0:xtdf'  # Input fast data source.
output_source = ''  # Output fast data source, empty to use same as input.

# CalCat parameters
creation_time = ""  # The timestamp to use with Calibration DB. Required Format: "YYYY-MM-DD hh:mm:ss" e.g. 2019-07-04 11:02:41
cal_db_interface = ''  # Not needed, compatibility with current webservice.
cal_db_timeout = 0  # Not needed, compatbility with current webservice.
cal_db_root = '/gpfs/exfel/d/cal/caldb_store'

# Operating conditions
mem_cells = 512  # Memory cells, LPD constants are always taken with 512 cells.
bias_voltage = 250.0  # Detector bias voltage.
capacitor = '5pF'  # Capacitor setting: 5pF or 50pF
photon_energy = 9.2  # Photon energy in keV.
category = 0  # Whom to blame.

# Correction parameters
offset_corr = True  # Offset correction.
rel_gain = True  # Gain correction based on RelativeGain constant.
ff_map = True  # Gain correction based on FFMap constant.
gain_amp_map = True  # Gain correction based on GainAmpMap constant.

# Output options
overwrite = True  # set to True if existing data should be overwritten
chunks_data = 1  # HDF chunk size for pixel data in number of frames.
chunks_ids = 32  # HDF chunk size for cellId and pulseId datasets.
create_virtual_cxi_in = out_folder  # Folder to create virtual CXI files in (for each sequence).

# Parallelization options
sequences_per_node = 1  # Sequence files to process per node
max_nodes = 8  # Maximum number of SLURM jobs to split correction work into
num_workers = 8  # Worker processes per node, 8 is safe on 768G nodes but won't work on 512G.
num_threads_per_worker = 32  # Number of threads per worker.

def balance_sequences(in_folder, run, sequences, sequences_per_node, karabo_da, max_nodes):
    from xfel_calibrate.calibrate import balance_sequences as bs
    return bs(in_folder, run, sequences, sequences_per_node, karabo_da, max_nodes=max_nodes)

Types of changes

• New feature (non-breaking change which adds functionality)

Checklist:

• My code follows the code style of this project.
• My change requires a change to the documentation.

Reviewers

@schmidtp @ahmedk

Thanks!
Cyril