fix(GH2): Avoid getting wrong data sources or including control devices' sources.

Description

This addresses this issue on sorting receiver names that can match the wrong receiver with the wrong DA.

Two issues are addressed here:

• Remove unneeded sort
• Add daqOutput to the receiver names by default to avoid getting ctrl device sources

receiver_template = "RECEIVER{}"
receiver_affixes = [""]
control_template = "CONTROL"

run_dc.select(receiver_names).all_sources
> frozenset({'SPB_50UM_GH2/DET/RECEIVER', 'SPB_50UM_GH2/DET/RECEIVER:daqOutput'})

run_dc.select('SPB_50UM_GH2/DET/RECEIVER').files

> [FileAccess('/gpfs/exfel/exp/SPB/202431/p900463/raw/r0053/RAW-R0053-GH2CTRL00-S00001.h5'),
 FileAccess('/gpfs/exfel/exp/SPB/202431/p900463/raw/r0053/RAW-R0053-GH2CTRL00-S00000.h5')]

Adding daqOutput will lead to removing error prone requirement in adding daqOutput for reciever names

e.g.

  SA2_XTD6_HIREX:
    detector-type: Gotthard2
    karabo-da:
    - GH200/1
    - GH200/2
    receiver-affixes:
    - _MASTER:daqOutput
    - _SLAVE:daqOutput
    receiver-template: GOTTHARD2{}
    control-template: GOTTHARD2_CTRL

  SPB_50UM_GH2:
    detector-type: Gotthard2
    karabo-da:
    - GH201
    receiver-affixes:
    - :daqOutput

How Has This Been Tested?

Relevant Documents (optional)

The report with the failed dark

Types of changes

• Bug fix (non-breaking change which fixes an issue)

Checklist:

• Update calibration configurations while deploying.

Reviewers

@schmidtp @kluyvert

changed the description

added 1 commit

• e08eb281 - fix: remove sort from dark nb and add hard code daqOutput channel in the...

Compare with previous version

changed title from fix(GH2): remove the unneeded sort for receiver names. to fix(GH2): Avoid getting wrong data sources or including control devices' sources.

added Update Calibration Configurations! label

changed the description

added 5 commits

• e08eb281...84cdbcf3 - 2 commits from branch test/new_test_runs
• 28157682 - fix(GH2): remove the unneeded sort for receiver names.
• 83787891 - fix: remove sort from dark nb and add hard code daqOutput channel in the...
• 2fe4ff72 - test: update test config

Compare with previous version

I think this is still not quite right. .all_sources is a set, so when you turn it into a list, the order is random! The sorted() is not really right either, but I guess happened to do the right thing with the 'master' & 'slave' names.

If we rely on a 1:1 correspondence between karabo_da and data_sources, let's do this more carefully, something like:

if len(receiver_affixes) != len(karabo_da):
    raise ValueError("recevier_affixes & karabo_da are not the same length")

da_to_source = {}
for da, aff in zip(karabo_da, receiver_affixes):
    sources = run_dc.select(f"*{receiver_template.format(aff)}:daqOutput").all_sources
    if len(sources) != 1:
        raise ValueError(f"No/multiple sources found for {aff!r}")
    da_to_source[da] = list(sources)[0]

The error messages could be better there, I kept it short to illustrate the idea.

added 1 commit

• 9a97afbb - fix: map_da_to_source function to avoid unsorted sources in a list

Compare with previous version

head scratch

So since we already got to a place where we need to change the configuration... just reviewing this MR made me confused again about all the different parmeters and how they are replaced or appended to each other. The names are just not that clear.

Can we try a single template/pattern parameter for control input, instrument input and instrument output, and then a single list to add the 25um complication? On top of the obligatory karabo_id of course:

karabo_id = 'FXE_XAD_G2XES'
ctrl_source_template = '{}/DET/CONTROL'  # getting rid of `control-template`
input_source_template = '{}/DET/RECEIVER_{}:daqOutput'  # example for HIREX
input_source_suffixes = ['MASTER', 'SLAVE']  # example for HIREX
output_source_template = '{}/CORR/RECEIVER:daqOutput

For 50um GH, input_source_suffixes can contain an empty string. For other creative namings for 25um out there, input_source_template and input_source_suffixes can be adapted as needed.

The explicit names above are only exemplary of course and replaceable.

added 1 commit

• 7d636390 - fix: move checking da/affixes numbers to corr notebook only

Compare with previous version

added 1 commit

• 383fc779 - refactor: update the input parameter names

Compare with previous version

added 1 commit

• adfbb9a2 - refactor: update the input parameter names

Compare with previous version

added 1 commit

• f131dd17 - fix: da_to_pdu dict modified

Compare with previous version

added 10 commits

• f131dd17...4eecbf4a - 3 commits from branch test/new_test_runs
• 530f93bc - fix(GH2): remove the unneeded sort for receiver names.
• 6b7ffe71 - fix: remove sort from dark nb and add hard code daqOutput channel in the...
• 03cf1450 - test: update test config
• 2fff513c - fix: map_da_to_source function to avoid unsorted sources in a list
• 9b368d82 - fix: move checking da/affixes numbers to corr notebook only
• 094d031b - refactor: update the input parameter names
• 0e65e4f8 - fix: da_to_pdu dict modified

Compare with previous version

added 1 commit

• a26b3b47 - Apply 1 suggestion(s) to 1 file(s)

Compare with previous version