This relies upon and demonstrates the change in !781 (merged). We get rid of the 'retrieve constants' pre notebook and look up the constants directly in the correction notebook instead. This relies on CalParrot recording the constant lookups for reproducibility, and the 'fragments' mechanism in !781 (merged) for recording constant metadata in the YAML file without risking corrupting it.
This reduces code duplication between the main & pre notebooks, so there's less risk of them getting out of sync. It's probably also slightly faster on average, because it needs one fewer Slurm job.
xfel-calibrate epix100 CORRECT \
--cal-db-timeout 300000 --cal-db-interface 'tcp://max-exfl016:8015#8044' \
--karabo-da EPIX02 --bias-voltage 200 --in-folder /gpfs/exfel/exp/MID/202301/p003346/raw \
--out-folder /gpfs/exfel/data/scratch/kluyvert/epix-correct-p3346-r86 \
--karabo-id MID_EXP_EPIX-2 --run 86The resulting metadata YAML file includes:
retrieved-constants:
EPIX02:
constants:
NoiseEPix100:
ccv_id: 160301
creation-time: '2023-03-30T19:37:45.000+02:00'
dataset: /ePix100_M18/NoiseEPix100/0
path: /gpfs/exfel/d/cal/caldb_store/xfel/cal/epix100-type/epix100_m18/cal.1680198257.124287.h5
OffsetEPix100:
ccv_id: 160300
creation-time: '2023-03-30T19:37:45.000+02:00'
dataset: /ePix100_M18/OffsetEPix100/0
path: /gpfs/exfel/d/cal/caldb_store/xfel/cal/epix100-type/epix100_m18/cal.1680198255.2475276.h5
RelativeGainEPix100:
ccv_id: 146898
creation-time: '2022-08-03T00:11:29.000+02:00'
dataset: /ePix100_M18/RelativeGainEPix100/0
path: /gpfs/exfel/d/cal/caldb_store/xfel/cal/epix100-type/epix100_m18/cal.1665416917.634143.h5
physical-name: ePix100_M18