Skip to content

[LPD] [Correct] Exclude trains with <4 modules from LPD VCXI files

Thomas Kluyver requested to merge lpd-vcxi-require-4-mods into master

Description

One LPD module (module 11) is generating some bogus train IDs, and its sequence file boundaries get out of sync with the other modules. When we create VCXI files 'by sequence', this means that the first train or two only has data from module 11 (and the last train or two are missing data from module 11). Most of the file is still OK, but users looking at the first 10 images in every file see missing data for 15 of 16 modules and understandably think it's broken. This change excludes trains with <4 modules (an arbitrary number) from the VCXI files, which should hide this issue.

Where this happens, there will still be some images with module 11 missing, because it's in the 'wrong' sequence file. Sequence files are not guaranteed to be aligned between different modules, so the whole concept of making a VCXI file for a batch of sequence files is kind of flawed anyway.

How Has This Been Tested?

Ran:

xfel-calibrate lpd CORRECT --slurm-mem 700 \
    --slurm-name correct_FXE_lpd_202201_p003073_r91 \
    --cal-db-timeout 300000 --cal-db-interface 'tcp://max-exfl016:8015#8044' \
    --karabo-da LPD00 LPD01 LPD02 LPD03 LPD04 LPD05 LPD06 LPD07 LPD08 LPD09 LPD10 LPD11 LPD12 LPD13 LPD14 LPD15 \
    --mem-cells 512 --bias-voltage 250.0 --capacitor 5pf --photon-energy 9.2 --category 0 \
    --offset-corr --rel-gain --ff-map --gain-amp-map \
    --in-folder /gpfs/exfel/exp/FXE/202201/p003073/raw --run 91 \
    --karabo-id FXE_DET_LPD1M-1 \
    --out-folder /gpfs/exfel/data/scratch/kluyvert/lpd-corr-p3073-r91-b \
    --create-virtual-cxi-in /gpfs/exfel/data/scratch/kluyvert/lpd-corr-p3073-r91-b/vcxi

Types of changes

  • Bug fix (non-breaking change which fixes an issue)

Checklist:

  • My code follows the code style of this project.

Reviewers

@dallanto @schmidtp

Edited by Thomas Kluyver

Merge request reports