[EPIX][CORR] Optimize histograms and plots

Description

Optimization of plots in ePix Correction Notebook, as stated in https://git.xfel.eu/calibration/planning/-/issues/167

How Has This Been Tested?

Run 55 from p003346 (EPIX-1 and EPIX-2).

Relevant Documents (optional)

Comparision between the generated reports before/after this MR, where the histpgrams are optimized and the plots now show meaningfull information:

OLD: MID_EXP_EPIX-1_correct_003346_r55_230330_195859.pdf

NEW:EPIX100CORRECTCalibration_new.pdf

Types of changes

• Bug fix (non-breaking change which fixes an issue)
• New feature (non-breaking change which adds functionality)

Specifically:

• Using np.histogram instead of xcal.HistogramCalculator to calculate histograms
• Using extra_geom instead of xana.heatmapPlot
• Dynamic colorbar limits, replacing the fixed ones
• Added additional plot for cluster corrected data
• Added meaningfull titles to the plots

Reviewers

@ahmedk

assigned to @duarten

added 1 commit

• 9c6e5dfb - improve plots and histograms

Compare with previous version

changed the description

added Waiting for review label

Ready for review

changed title from Draft: [EPIX][CORR] to Draft: [EPIX][CORR] Optimize histograms and plots

marked this merge request as ready

mentioned in merge request !834 (merged)

added 1 commit

• 3f2177d9 - skip singles plot if empty

Compare with previous version

New commit to solve the same issue as in !834 (merged). Ready for review again

resolved all threads

added 1 commit

• 79a57f4d - solve merge conflicts after !835 (merged)

Compare with previous version

changed milestone to %3.11.0

LGTM.

Note for later deployment tests: This MR leads to changing the values for patterns and pixels_classified. The reason is that gain_cnst value is computed differently now

Hello Nuno, Could you document here the changes fro gain_cnst? It is not mentioned in the MR description and it is important to have the reason for such change documented somewhere as this affects the data values.

removed milestone %3.11.0

changed milestone to %3.11.1

changed milestone to %3.11.0

Of course.

It was not mentioned in the description of this MR, but it's mentioned in the comments in the code above the line where gain_cnst is defined:

# gain constant is given by the mode of the gain map because all bad pixels are masked using this value

Up until now, we have been calculating gain_cnst by looking the median of the gain map.

However, the actual gain constant value is calculated in the FF analysis notebook (!750 (merged), which is still not merged to pycalibration), and is not directly outputed, since this NB only injects to the DB the relative gain map after all the processing steps. However, because this map is actually an absolute gain map, and because during its processing the bad pixels are masked with the calculated gain constant, it's value can be found by looking at the bad pixels.

And this is slightly different from the median value of the map. The difference is not great:

>>> gain_cnst = np.median(const_data)
>>> print(f'Old gain constant: {1/gain_cnst:.3f} ADU/keV')
Old gain constant: 16.079 ADU/keV
>>>
>>> _vals,_counts = np.unique(const_data, return_counts=True)
>>> gain_cnst = _vals[np.argmax(_counts)]
>>> print(f'New gain constant: {1/gain_cnst:.3f} ADU/keV')
New gain constant: 16.032 ADU/keV

But since it is virtually effortless, we should use the most accurate value.

So I just find the most commom value of the map, which will for sure correspond to the value passed to the bad pixels. In fact, it's the only value that ever repeats:

>>> np.sort(_counts)
array([   1,    1,    1, ...,    1,    1, 4596])

I understand this change is out of the scope of this MR. Should I suggest it in a seperate MR and keep this as was in here?