Skip to content

[ePix100] [Correct] Remove pre notebook

Thomas Kluyver requested to merge feat/rm-pre-epix100 into master

Description

This relies upon and demonstrates the change in !781 (merged). We get rid of the 'retrieve constants' pre notebook and look up the constants directly in the correction notebook instead. This relies on CalParrot recording the constant lookups for reproducibility, and the 'fragments' mechanism in !781 (merged) for recording constant metadata in the YAML file without risking corrupting it.

This reduces code duplication between the main & pre notebooks, so there's less risk of them getting out of sync. It's probably also slightly faster on average, because it needs one fewer Slurm job.

How Has This Been Tested?

xfel-calibrate epix100 CORRECT \
  --cal-db-timeout 300000 --cal-db-interface 'tcp://max-exfl016:8015#8044' \
  --karabo-da EPIX02 --bias-voltage 200 --in-folder /gpfs/exfel/exp/MID/202301/p003346/raw \
  --out-folder /gpfs/exfel/data/scratch/kluyvert/epix-correct-p3346-r86 \
  --karabo-id MID_EXP_EPIX-2 --run 86

The resulting metadata YAML file includes:

retrieved-constants:
  EPIX02:
    constants:
      NoiseEPix100:
        ccv_id: 160301
        creation-time: '2023-03-30T19:37:45.000+02:00'
        dataset: /ePix100_M18/NoiseEPix100/0
        path: /gpfs/exfel/d/cal/caldb_store/xfel/cal/epix100-type/epix100_m18/cal.1680198257.124287.h5
      OffsetEPix100:
        ccv_id: 160300
        creation-time: '2023-03-30T19:37:45.000+02:00'
        dataset: /ePix100_M18/OffsetEPix100/0
        path: /gpfs/exfel/d/cal/caldb_store/xfel/cal/epix100-type/epix100_m18/cal.1680198255.2475276.h5
      RelativeGainEPix100:
        ccv_id: 146898
        creation-time: '2022-08-03T00:11:29.000+02:00'
        dataset: /ePix100_M18/RelativeGainEPix100/0
        path: /gpfs/exfel/d/cal/caldb_store/xfel/cal/epix100-type/epix100_m18/cal.1665416917.634143.h5
    physical-name: ePix100_M18

Types of changes

  • Refactor (refactoring code with no functionality changes)

Checklist:

  • My code follows the code style of this project.

Reviewers

@ahmedk @schmidtp

Merge request reports