Skip to content
Snippets Groups Projects

Add support to splice Influx-derived data into raw data

Open Philipp Schmidt requested to merge feat/splice-influx-data into master

Data imported from Influx via https://git.xfel.eu/amore/influx2h5 can be hard to integrate into existing raw data, as different files may not contain data for the same source and train, no matter whether they contain duplicate or distinct keys.

This MR adds a reduction plugin that can properly integrate such Influx-derived data, either by simply replacing the data for any key retrieved from Influx or removing any affected source entirely from the original files and moving them to their own aggregator.

Two new reduction instruction are used to implement this:

  • set-aggregator allows to override the aggregator a source is written to.
  • replace-keys allows to replace a key's data from the input collection with data from another location

Merge request reports

Loading
Loading

Activity

Filter activity
  • Approvals
  • Assignees & reviewers
  • Comments (from bots)
  • Comments (from users)
  • Commits & branches
  • Edits
  • Labels
  • Lock status
  • Mentions
  • Merge request status
  • Tracking
  • Loading
Please register or sign in to reply
Loading