[Jungfrau] Add Gain setting to Jungfrau notebooks

Description

This MR brings gain settings to Jungfrau constants, allowing to differentiate between different operating conditions.

The gain settings are saved in RUN section of files as settings and are:

dynamicgain: G0, G1, G2
dynamichg0: HG0, G1, G2
fixgain1: G1 
fixgain2: G2 
forceswitchg1
forceswitchg2

The expected configurations are then, for dark notebooks:

Runs taken with dynamicgain, forceswitchg1, forceswitchg2 -> gain_setting = 0
Runs taken with dynamichg0, forceswitchg1, forceswitchg2 -> gain_setting = 1

and for correction:

settings == dynamicgain -> gain_setting = 0
settings == dynamichg0 -> gain_setting = 1

How Has This Been Tested?

I tested the creation of local constants, but I did not test injection and retrieval to/from the database.

Relevant Documents (optional)

Types of changes

• New feature (non-breaking change which adds functionality)

Checklist:

• My code follows the code style of this project.
• My change requires a change to the documentation.
• I have updated the documentation accordingly.

Reviewers

@calibration

@mramilli FYI.

unmarked as a Work In Progress

changed title from WIP: Add Gain setting to Jungfrau notebooks to [Jungfrau] Add Gain setting to Jungfrau notebooks

changed the description

added 1 commit

• 1e11a36b - Map detector setting to operating condition using enums

Compare with previous version

resolved all threads

I like enums. I'm a bit confused by the overlapping values. It seems that the first one name for a given value is what you get when instantiating for the value, but does having somewhat "hidden" names like forceswitchg1 etc. values do anything for us?

An enumeration is a set of symbolic names (members) bound to unique, constant values. Within an enumeration, the members can be compared by identity, and the enumeration itself can be iterated over.

It seems that the first one name for a given value is what you get when instantiating for the value

So the first one name is the string for dynamic gain (or used for high gain as well.)

A dark is created from 3 runs of 3 different strings (for each gain stage.). In principle only dynamicgain or dynamichg0 are going to be corrected and the rest of the strings are for runs that are used for generating dark constants.

But the non-unique values here are used to be able to still correct the medium and low gain runs with the corresponding constants, if requested.

if this is the explanation, then this class is wrong. Darks for gain_setting == 0 will be generated with:

• dynamicgain for high gain
• forceswitchg1 for medium gain
• forceswitchg2 for low gain Darks for gain_setting == 1 will be using:
• dynamichgo for high gain
• forceswitchg1 for medium gain
• forceswitchg2 for low gain

Fixgain1 and fixgain2 will probably require a different gain_setting value. And as for forceswitchg1 and forceswitchg2, I agree that the eventuality that the respective runs are going to be offset subtracted and gain corrected is low, but in case, since they are used for both gain settings, they should be able to use either one, whichever is closest in time.

yes, true. After checking again the description I realized that I was mistaken. Thanks @mramilli

The issue here is if a run of forceswitchg1 was requested then we can't (with the current system) retrieve a constant of either gain_setting = 1 or 0.

Maybe we create an enum indeed for dynamicgain and dynamichgo only and the high gain run is the run that decides the condition for gain_setting value for the dark constants.

And in the case if a correction was requested for a low or medium gain run we retrieve for the moment a constant with gain_setting = 0.

Later if we are able to use get_possible_conditions we can retrieve a constant with gain_setting 0 or 1 for a run of a medium or low gain.

Does this make sense?

I this makes sense. Correcting a forceswitchg1 or forceswitchg2 run makes sense only for debugging purposes, so it would be only a minority of cases, or a mistake from some user clicking on the wrong button (but in the latter it is fine if a very old set of constants is retrieved).

I'm a bit confused by this code & discussion.

It looks a bit like we might be using an enum in a strange way. An enum is normally a straightforward definition of all the possible options for some value. So dynamicgain and forceswitchg1 would have different values (unless they are really just two names for the exact same thing).

Maybe what we want here are two enums - one with 6 values, one with 2 - and a method to go from the first to the second?

I can clarify only on the detector aspect, which can understandably be a bit confusing. There is one property in the JUNGFRAU control device called settings, and, as you pointed out, it has 6 possible values: dynamicgain, dynamichg0, forceswitchg1, forceswitchg2, fixgain1 and fixgain2 (we can forget about the last two for now).

dynamicgain and dynamichg0 are the values used to configure the detector in dynamic gain switching mode, with two different values of amplification for the first stage only.

The confusion arises when taking dark runs; to take a dark run for each gain stage the solution is:

• dynamicgain for high gain
• forceswitchg1 for medium gain
• forceswitchg2 for low gain

the two last values are used only for dark runs, and the three of them are processed together to create a set of offsets for when the detector operates with the property settings == dynamicgain, and this has been flagged in the DB by setting the value of gain_setting == 0.

Now, users want to be able to use dynamichg0. For technical reasons, this operation mode changes only the first gain stage, so for offset calculation we will have three runs as in the previous case, but:

• dynamichg0 for high gain
• forceswitchg1 for medium gain
• forceswitchg2 for low gain

The general idea is to have the constants produced by the notebook inserted in the DB with gain_settings == 1, and they will be used to correct data acquired with the property settings == dynamichg0.

This leave open the question on what to do if anybody tries to process a run acquired with settings == forceswitchg1.

changed this line in version 3 of the diff

Thanks for the sanity check!

From the discussion above, it transpires that we should create and use constants with gain_setting == 1 only when dynamichg0 is set.

As such, the enums are not needed, but a simple boolean check can be used instead.
I've implemented this and tested against runs provided by Thomas P. of HED.

When setting the Jungfrau_dark_analysis_all_gains_burst_mode_NBC notebooks with the following:

in_folder = '/gpfs/exfel/exp/HED/202131/p900227/raw'
run_high = 17 # run number for G0 dark run, required  # dynamichg0
run_med = 18 # run number for G1 dark run, required
run_low = 20 # run number for G2 dark run, required

then the constant's gain_setting is set to 1.

When set with the default notebook's runs, then the gain_setting is set to 0.

For correction, the same logic is applied to retrieve constants.

This was tested against the test calibration database, where constants were injected and retrieved as expected with the new gain_setting == 1.

added 1 commit

• 6497501f - Create and use constants with gain_setting 1 if dynamichg0 is set

Compare with previous version

added 1 commit

• 5398e7b3 - Create and use constants with gain_setting 1 if dynamichg0 is set

Compare with previous version

LGTM

It's been running for a day or so and I did not hear of any problems.
Thanks everyone for the discussions!

mentioned in commit b8eafa40

merged

changed milestone to %3.4.2