Add integration time as AGIPD operation condition to all notebooks

Description

pycalibration part of https://git.xfel.eu/gitlab/calibration/planning/issues/47, adding integration time of AGIPD pixels to the detector condition to include it in future constants.

How Has This Been Tested?

xfel-calibrate agipd DARK --slurm-scheduling 10 --request-time 2021-07-01T08:10:38 --slurm-name dark_SPB_agipd_integration_time_test --report-to ./report --cal-db-interface tcp://max-exfl016:8015#8044 --db-output --concurrency-par karabo_da --karabo-da AGIPD00 AGIPD01 AGIPD02 AGIPD03 AGIPD04 AGIPD05 AGIPD06 AGIPD07 AGIPD08 AGIPD09 AGIPD10 AGIPD11 AGIPD12 AGIPD13 AGIPD14 AGIPD15 --receiver-id {}CH0 --karabo-id-control SPB_IRU_AGIPD1M1 --karabo-da-control AGIPD1MCTRL00 --h5path-ctrl /CONTROL/{}/MDL/FPGA_COMP --mem-cells 0 --thresholds-offset-hard 0 0 --thresholds-offset-hard-hg 3500 6000 --thresholds-offset-hard-mg 6000 9000 --thresholds-offset-hard-lg 6000 9000 --in-folder /gpfs/exfel/exp/SPB/202130/p900188/raw --out-folder ./out --karabo-id SPB_DET_AGIPD1M-1 --operation-mode ADAPTIVE_GAIN --run-high 523 --run-med 524 --run-low 525

xfel-calibrate agipd CORRECT --slurm-scheduling 80 --slurm-mem 750 --request-time 2021-06-24T12:17:13 --slurm-name correct_MID_agipd_202002_p002616_r12 --report-to ./report --cal-db-timeout 300000 --cal-db-interface tcp://max-exfl016:8015#8044 --karabo-da AGIPD00 AGIPD01 AGIPD02 AGIPD03 AGIPD04 AGIPD05 AGIPD06 AGIPD07 AGIPD08 AGIPD09 AGIPD10 AGIPD11 AGIPD12 AGIPD13 AGIPD14 AGIPD15 --receiver-id {}CH0 --karabo-id-control MID_EXP_AGIPD1M1 --karabo-da-control AGIPD1MCTRL00 --h5path-ctrl /CONTROL/{}/MDL/FPGA_COMP --overwrite --sequences-per-node 1 --bias-voltage 300 --force-hg-if-below --hg-hard-threshold 1000 --force-mg-if-below --mg-hard-threshold 1000 --blc-stripes --common-mode --cm-dark-range -30 30 --cm-dark-fraction 0.15 --cm-n-itr 4 --rel-gain --adjust-mg-baseline --blc-set-min --low-medium-gap --ff-gain 7.2 --in-folder /gpfs/exfel/exp/MID/202002/p002616/raw --out-folder ./out --karabo-id MID_DET_AGIPD1M-1 --run 12 --priority 2

Types of changes

• Breaking change (fix or feature that would cause existing functionality to not work as expected)

Checklist:

• My code follows the code style of this project.

Reviewers

@calibration

added In Progress label

added 1 commit

• 68ae310e - Add AGIPD integration time condition to cal_tools tests

Compare with previous version

The technical details of this MR itself look fine to me. I don't have a good picture of how it will interact with data & constants created before the condition was added, though.

Thanks for the review.

Concerning constant interaction, in the current state of things we would not be able to retrieve constants taken before integration time is added even if they are physically suitable (i.e. we are measuring with the legacy value of 12 cycles). The immediate workaround we'll take in coordination with Jola is to continue not injecting an integration time whenever the data has been taken with the 12 cycles. The disadvantage here is that we cannot include 12 when evaluating acceptable operation condition deviations.

For a proper fix, we can inject an integration time of 12 for all prior constants. And even better fix is to expand the query options to account for such situations when talking to CalCat.

added 1 commit

• 3ae220d7 - Pass AgipdCorrections.initialize_from_db arguments by keyword

Compare with previous version

resolved all discussions

added 1 commit

• 0d696e74 - Minor fixes in AGIPD dark notebook and agipblib

Compare with previous version

added 1 commit

• eec64962 - Bump iCalibrationDB version to 2.0.9

Compare with previous version

@jsztuk mentioned that it would be nice to have integration_time exposed in the first cell, same as the other parameters. This can be useful in case of exceptional cases of testing or issues with a non-stored parameter for useful data.

added 1 commit

• 0a995d30 - Add notebook parameter for AGIPD integration time to override detection

Compare with previous version

added 1 commit

• 26063376 - Override integration time condition deviations when injecting FF constants

Compare with previous version

added Waiting for review label and removed In Progress label

unmarked as a Work In Progress

changed the description

LGTM

changed milestone to %3.4.1

LGTM! I like how discrete the changes are.

Thanks for the reviews, let's go!

merged

mentioned in commit 7d16cf6b

removed Waiting for review label