Skip to content

25 μm interleaving support for GOTTHARD-II

David Hammer requested to merge gh2-25um-interleave into master

Following discussion in https://git.xfel.eu/calibration/planning/-/issues/164, I've extended the array stacking capability of the ShmemTrainMatcher to support interleaving. In terms of data movement, interleaving does the same thing as stacking except two axes are "merged" (so the output shape looks like the result of np.concatenate rather than of np.stack). That's not very important as it's done in a custom way anyway.

To do:

  • test a lot more
  • support for key groups
  • maybe encapsulate the stacking parts
  • make naming more consistent - should we go back to "merge" as superset of interleaving and stacking?

So far, I've tested with two GH2 sources (using one FXE and one SPB because I need different source names in the matcher). With the following "Array stacking" row:

  • Select: True
  • Source pattern: (FXE_XAD_G2XES|SPB_50UM_GH2)/DET/RECEIVER:daqOutput
  • Key pattern: data.adc
  • Replacement: PRETEND_25UM_GH2
  • Group type: sources
  • Merge method: interleave
  • Axis: 1

I glance at the bridge and see the shapes I'd expect:

Train ID: 14118 --------------------------
Data from 3 sources, REQ-REP took 0.03 ms

Source 1: 'FXE_XAD_G2XES/DET/RECEIVER:daqOutput' @ 14118
timestamp: 2023-02-22 13:59:29 (1677070769.581713) | delay: 93.79 ms
data:
 - [bool] corrections.badPixels, False
 - [bool] corrections.gain, False
 - [bool] corrections.lut, False
 - [bool] corrections.offset, False
 - [ndarray] data.bunchId, uint64, (2700,)
 - [ndarray] data.frameNumber, uint64, (2700,)
 - [ndarray] data.gain, uint8, (2700, 1280)
 - [ndarray] data.memoryCell, uint8, (2700,)
 - [ndarray] data.timestamp, float64, (2700,)
 - [int] data.trainId, 1441838433
 - [str] metadata.source, FXE_XAD_G2XES/DET/RECEIVER:daqOutput
 - [int] metadata.timestamp.tid, 1441838433
metadata:
 - [list] ignored_keys, []
 - [str] source, FXE_XAD_G2XES/DET/RECEIVER:daqOutput
 - [float] timestamp, 1677070769.581713
 - [str] timestamp.frac, 581713029000000000
 - [str] timestamp.sec, 1677070769
 - [int] timestamp.tid, 14118

Source 2: 'PRETEND_25UM_GH2' @ 14118
timestamp: 2023-02-22 13:59:29 (1677070769.581713) | delay: 93.79 ms
data:
 - [ndarray] data.adc, float32, (2700, 2560)
metadata:
 - [list] ignored_keys, []
 - [str] source, PRETEND_25UM_GH2
 - [float] timestamp, 1677070769.581713
 - [str] timestamp.frac, 581713029000000000
 - [str] timestamp.sec, 1677070769
 - [int] timestamp.tid, 14118

Source 3: 'SPB_50UM_GH2/DET/RECEIVER:daqOutput' @ 14118
timestamp: 2023-02-22 13:59:29 (1677070769.637645) | delay: 37.86 ms
data:
 - [bool] corrections.badPixels, False
 - [bool] corrections.gain, False
 - [bool] corrections.lut, False
 - [bool] corrections.offset, False
 - [ndarray] data.bunchId, uint64, (1200,)
 - [ndarray] data.frameNumber, uint64, (1200,)
 - [ndarray] data.gain, uint8, (1200, 1280)
 - [ndarray] data.memoryCell, uint8, (1200,)
 - [ndarray] data.timestamp, float64, (1200,)
 - [int] data.trainId, 1599789511
 - [str] metadata.source, SPB_50UM_GH2/DET/RECEIVER:daqOutput
 - [int] metadata.timestamp.tid, 1599789511
metadata:
 - [list] ignored_keys, []
 - [str] source, SPB_50UM_GH2/DET/RECEIVER:daqOutput
 - [float] timestamp, 1677070769.637645
 - [str] timestamp.frac, 637645804000000000
 - [str] timestamp.sec, 1677070769
 - [int] timestamp.tid, 14118
Edited by David Hammer

Merge request reports