MID requested some time ago to round the result of AGIPD corrections to absolute number of photons. Assuming the intensity is in units of keV after proper corrections, this is simply a division by the actual photon energy. In certain experimental situations this can improve SNR, reduce the data volume and is done at MID on the analysis-side regularly.
The photon energy can be taken from a DoocsXGM device via the use_xgm_device parameter. It is checked for a 1% tolerance and photon rounding is disabled if this fails. If the XGM device is not available, the photon-energy operating condition is used instead with a warning. After rounding, any values below 0 are set to 0 and masked as BadPixels.VALUE_OUT_OF_RANGE.
To exploit the potential data reduction opportunities, this MR contains two additional changes:
Make the compressed fields configurable via parameter
Add code to cast the image data in-place to a different (smaller or equal) type
These two changes allow for pretty impressive results with almost zero cost, assuming you want rounded photons:
treatment
type
compressed
size
read*
standard
f4
4.2G (100%)
4.45s (100%)
standard
f4
X
3.6G (86%)
13.5s (303%)
rounded
f4
4.2G (100%)
4.46s (100%)
rounded
f4
X
403M (10%)
12.2s (274%)
rounded
u2
2.2G (52%)
2.6s (58%)
rounded
u2
X
153M (4%)
4.33s (97%)
* Time required for xd.H5File(<one sequence>)['MID_DET_AGIPD1M-1/DET/15CH0:xtdf', 'image.data'].ndarray(), 7 runs warm-up, 7 runs average measurement
When data is casted to an integer-type, any value exceeding its bounds are set to the respectively min/max value and marked as BadPixels.VALUE_OUT_OF_RANGE.
