Frame filtering by candidate list

This implements the frame filtering by the candidate list with formats that we discussed: h5, csv. In both formats we expect three datasets: trainId, pulseId, classId, which can be given under any name (parameter). The third options is EXDF format, which also can be read as h5 but it is read here with extra-data and due to that can be split on sequence files

@schmidtp @kluyvert @turkot

added 2 commits

• f8a006cd - Support csv format, isolate file reading
• 95cc758a - Fix pulse alignment

Compare with previous version

marked this merge request as ready

I'll leave the details to Oleksii, but looks really nice overall. Thanks!

Very nice implementation indeed, and all my comments except the first one are very minor. I'll run the manual tests now and afterwards should be good to merge.

assigned to @esobolev

requested review from @turkot

added 1 commit

• 6f6b8452 - Apply suggestions

Compare with previous version

added 1 commit

• 0b00336c - Add option to choose which frame classes to drop

Compare with previous version

added 6 commits

• 2019d066 - 1 commit from branch feat/legacy
• 489661ee - Add reduction of frames by candidate list
• 515d1d73 - Support csv format, isolate file reading
• 5eea8ab0 - Fix pulse alignment
• 5b268025 - Apply suggestions
• 1fb1e9d8 - Add option to choose which frame classes to drop

Compare with previous version

added 1 commit

• d2e0a17f - Allow --framelist-train-id-key and --framelist-pulse-id-key to be optional...

Compare with previous version

Hi Egor, with !26 (d2e0a17f) I would like to suggest to make --framelist-train-id-key and --framelist-pulse-id-key parameters optional also for h5 and csv file inputs. Please let me know and feel free to discard or modify this commit in case you don't like the idea.

added 1 commit

• f9203a0e - Apply 1 suggestion(s) to 1 file(s)

Compare with previous version

resolved all threads

added 1 commit

• f0121cdf - Apply 1 suggestion(s) to 1 file(s)

Compare with previous version

added 1 commit

• f522c1c6 - Disallow empty list in the --framelist-drop-classes option

Compare with previous version

LGTM - code passed my tests with exdf, h5, and csv files. I believe it is ready to be merged.

Thanks for review, @turkot! Before we merge, let's wait for @schmidtp to finish the review of !25

approved this merge request

Suggestions for the arguments descriptions.

Thanks, @turkot!

requested changes

added 1 commit

• ed8c063f - Fix help strings of argument parser

Compare with previous version

approved this merge request