[LPD] New notebook for injecting constants from stored h5files

This MR is related to https://git.xfel.eu/gitlab/detectors/cal_db_interactive/merge_requests/70

Description

This MR introduces a notebook to inject calibration constants from h5files.

It is planned to be used for LPD to inject 4 gain constants to the database.

Constant names:
1- FFMap
2- BadPixelsFF
3- RelativeGain
4- GainAmpMap

The expected structure for the stored constants is: in_folder/{constant_name}_{karabo_da}.h5

The expected data path for the constant array inside the h5file is: ['constant_name']

There is a change as well for the get_report(__) function in cal_tools to have the ability in injecting a report path metadata even if there is no report (i.e. no_report/<injection-date>).

How Has This Been Tested?

xfel-calibrate LPD INJECT_CONSTANTS \
--constant-names BadPixelsFF FFMap RelativeGain GainAmpMap \
--in-folder /gpfs/exfel/data/scratch/yousefh/LPD_CalinConst_Provisional/ \
--creation-time 2021-04-21T13:06:00 \
--karabo-id FXE_DET_LPD1M-1 \
--out-folder /gpfs/exfel/data/scratch/ahmedk/test/LPD_constants3 \
--cal-db-interface tcp://max-exfl017:8015#8040

Relevant Documents (optional)

Types of changes

• New feature (non-breaking change which adds functionality)

Checklist:

• My code follows the code style of this project.
• My change requires a change to the documentation.
• I have updated the documentation accordingly.

Reviewers

@calibration

changed the description

added 1 commit

• 103fcf85 - add multiprocessing

Compare with previous version

added 1 commit

• 752babb2 - update notebook after injecting constants to the database

Compare with previous version

added 1 commit

• a7407658 - add cal_db_interactive tag

Compare with previous version

unmarked as a Work In Progress

changed the description

added 1 commit

• 3dce9494 - validate input constant_names

Compare with previous version

changed the description

added 1 commit

• 965d39ef - validate input constant_names

Compare with previous version

changed the description

added 1 commit

• a7b702c4 - reduce processes

Compare with previous version

changed title from New notebook for injecting constants from stored h5files to [LPD] New notebook for injecting constants from stored h5files

added 1 commit

• 7726096d - update cal_db_interactive tag

Compare with previous version

I think it's good

The notebook looks good.

Given that the constants are generated somewhere else, the notebook is essentially a CLI tool to talk to CalCat, right?

Anyway, LGTM.

Yes, that's right: The expected input is a path to a HDF5 file, which is then copied to our temporary folder on GPFS. The info is sent to CalCat, which then itself takes care of copying the file over to its permanent location, storing other metadata in the database.

I'll take care of rebasing, and I'll merge :)

added 17 commits

• 7726096d...2d9b56b8 - 16 commits from branch master
• 57b1b82c - Rebase from master

Compare with previous version

enabled an automatic merge when the pipeline for 57b1b82c succeeds