Skip to content

[LPD][Correct] Fix/constant dtype conversion

Karim Ahmed requested to merge fix/constant_dtype_conversion into master

Description

Before !867 (merged) and !779 (merged), constant arrays were loaded in the notebook and were initialized in the correct type for bad pixels.

After using CalibrationData API, https://git.xfel.eu/calibration/pycalibration/-/blob/master/src/cal_tools/calcat_interface.py#L626-638. Masks were stored in the wrong expected values for the corrected data.

with h5py.File(self.caldb_root / mdata["path"], "r") as cf:
    shape = cf[f"{dataset}/data"].shape
    dtype = cf[f"{dataset}/data"].dtype

As there are some old BadPixels constants that are stored as <f8 with NANs. When converting these nans to uint32, instead of being converted to 0 uint32 it gets converted to 2147483648

For the BadPixels, I convert NANs to zeros first. This bug was caught using the automated tests and the masks were the only non-validated data.

How Has This Been Tested?

Relevant Documents (optional)

Types of changes

  • Bug fix (non-breaking change which fixes an issue)

Checklist:

Reviewers

@kluyvert

Edited by Karim Ahmed

Merge request reports