25 μm interleaving support for GOTTHARD-II
Following discussion in https://git.xfel.eu/calibration/planning/-/issues/164, I've extended the array stacking capability of the ShmemTrainMatcher
to support interleaving.
In terms of data movement, interleaving does the same thing as stacking except two axes are "merged" (so the output shape looks like the result of np.concatenate
rather than of np.stack
).
That's not very important as it's done in a custom way anyway.
To do:
- test a lot more
- support for key groups
- maybe encapsulate the stacking parts
- make naming more consistent - should we go back to "merge" as superset of interleaving and stacking?
So far, I've tested with two GH2 sources (using one FXE and one SPB because I need different source names in the matcher). With the following "Array stacking" row:
- Select:
True
- Source pattern:
(FXE_XAD_G2XES|SPB_50UM_GH2)/DET/RECEIVER:daqOutput
- Key pattern:
data.adc
- Replacement:
PRETEND_25UM_GH2
- Group type:
sources
- Merge method:
interleave
- Axis:
1
I glance at the bridge and see the shapes I'd expect:
Train ID: 14118 --------------------------
Data from 3 sources, REQ-REP took 0.03 ms
Source 1: 'FXE_XAD_G2XES/DET/RECEIVER:daqOutput' @ 14118
timestamp: 2023-02-22 13:59:29 (1677070769.581713) | delay: 93.79 ms
data:
- [bool] corrections.badPixels, False
- [bool] corrections.gain, False
- [bool] corrections.lut, False
- [bool] corrections.offset, False
- [ndarray] data.bunchId, uint64, (2700,)
- [ndarray] data.frameNumber, uint64, (2700,)
- [ndarray] data.gain, uint8, (2700, 1280)
- [ndarray] data.memoryCell, uint8, (2700,)
- [ndarray] data.timestamp, float64, (2700,)
- [int] data.trainId, 1441838433
- [str] metadata.source, FXE_XAD_G2XES/DET/RECEIVER:daqOutput
- [int] metadata.timestamp.tid, 1441838433
metadata:
- [list] ignored_keys, []
- [str] source, FXE_XAD_G2XES/DET/RECEIVER:daqOutput
- [float] timestamp, 1677070769.581713
- [str] timestamp.frac, 581713029000000000
- [str] timestamp.sec, 1677070769
- [int] timestamp.tid, 14118
Source 2: 'PRETEND_25UM_GH2' @ 14118
timestamp: 2023-02-22 13:59:29 (1677070769.581713) | delay: 93.79 ms
data:
- [ndarray] data.adc, float32, (2700, 2560)
metadata:
- [list] ignored_keys, []
- [str] source, PRETEND_25UM_GH2
- [float] timestamp, 1677070769.581713
- [str] timestamp.frac, 581713029000000000
- [str] timestamp.sec, 1677070769
- [int] timestamp.tid, 14118
Source 3: 'SPB_50UM_GH2/DET/RECEIVER:daqOutput' @ 14118
timestamp: 2023-02-22 13:59:29 (1677070769.637645) | delay: 37.86 ms
data:
- [bool] corrections.badPixels, False
- [bool] corrections.gain, False
- [bool] corrections.lut, False
- [bool] corrections.offset, False
- [ndarray] data.bunchId, uint64, (1200,)
- [ndarray] data.frameNumber, uint64, (1200,)
- [ndarray] data.gain, uint8, (1200, 1280)
- [ndarray] data.memoryCell, uint8, (1200,)
- [ndarray] data.timestamp, float64, (1200,)
- [int] data.trainId, 1599789511
- [str] metadata.source, SPB_50UM_GH2/DET/RECEIVER:daqOutput
- [int] metadata.timestamp.tid, 1599789511
metadata:
- [list] ignored_keys, []
- [str] source, SPB_50UM_GH2/DET/RECEIVER:daqOutput
- [float] timestamp, 1677070769.637645
- [str] timestamp.frac, 637645804000000000
- [str] timestamp.sec, 1677070769
- [int] timestamp.tid, 14118
Edited by David Hammer