[JUNGFRAU] [Dark] Fix bad pixel constant from darks taken in burst mode

Description

Corrected JUNGFRAU data in burst mode (16 cells) was getting a mask which was as expected for the first memory cell used, but in the other 15 cells, every pixel was marked with WRONG_GAIN_VALUE. The mask is used for the ROI reduction, which was therefore giving nan.

The issue is in the BadPixelsDark10Hz constants. For medium & low gains in burst mode, all but the first memory cell are marked as invalid. Then a check for the gain stage of each pixel gets nan, because there's no data, so it thinks all pixels are in the wrong gain stage. Finally, the code added in !914 (merged) 'reflects' WRONG_GAIN_VALUE into the high gain stage, where we do have valid data.

The solution here is to skip the processing entirely when no data is found for a memory cell. Where this is the case, I set the bad pixel constant to NO_DARK_DATA. It also reports which cells are missing.

How Has This Been Tested?

Run the dark notebook interactively, working with p6616 runs 272, 273 & 274 (JNGFR02) and checked the resulting constants against the existing ones from this report.

• The offset constant is the same in high gain (offset[..., 0]) and the same for the first memory cell used (offset[:, :, 15]).
• For the other cells in medium & low gain (offset[:, :, :15, 1:]), the offset changes from nan to 0. I can put it back to nan if we prefer. changed to preserve the nan values.
• The cells in medium & low gain for all but the first memory cell used (badpx[:, :, :15, 1:]) go from bad pixels WRONG_GAIN_VALUE | OFFSET_NOISE_EVAL_ERROR to now NO_DARK_DATA | NOISE_OUT_OF_THRESHOLD | OFFSET_OUT_OF_THRESHOLD, because of 0s replacing NaNs. NO_DARK_DATA | OFFSET_NOISE_EVAL_ERROR.

Types of changes

• Bug fix (non-breaking change which fixes an issue)

Checklist:

• My code follows the code style of this project.

Reviewers

@ahmedk @schmidtp

.gitlab-ci.yml

@@ -67,7 +67,7 @@ automated_test:
     - export LANG=C  # Hopefully detect anything relying on locale
     - python3 -m pip install ".[automated_test]"
     - echo "Running automated test. This can take sometime to finish depending on the test data."
-    - echo "Given variables are REFERENCE=$REFERENCE, OUTPUT=$OUTPUT, DETECTOR=$DETECTORS, CALIBRATION=$CALIBRATION"
+    - echo "Given variables are REFERENCE=$REFERENCE, OUTPUT=$OUTPUT, DETECTORS=$DETECTORS, CALIBRATION=$CALIBRATION"
     - python3 -m pytest ./tests/test_reference_runs --color yes --verbose --release-test --reference-folder /gpfs/exfel/data/scratch/xcaltst/test/$REFERENCE --out-folder /gpfs/exfel/data/scratch/xcaltst/test/$OUTPUT  --detectors $DETECTORS --calibration $CALIBRATION  --find-difference
   timeout: 24 hours