Skip to content

Add blocklist pattern when copying untouched files in webservice

Philipp Schmidt requested to merge feat/untouched_copy_blacklist into master

Description

Right now, any file not explicitly consumed by a correction (through the DAs configured in the data-mapping section of calibration_config) is copied over to the proc directory. While I'm pushing to longer do this in general, this creates an odd situation when we'd want to disable a particular detector correction (which might happen in case of LPD).

This MR introduces a blocklist pattern in copy_untouched_files to never copy over files from AGIPD, LPD or DSSC even if they end up in the untouched file list.

How Has This Been Tested?

>>> p = re.compile(r'\S*RAW-R\d{4}-(AGIPD|LPD|DSSC)\d{2}-S\d{5}.h5$')
>>> p.match('/gpfs/exfel/exp/FXE/201930/p900063/raw/r0844/RAW-R0844-DA01-S00008.h5')  # Taken from webservice log
>>> p.match('/gpfs/exfel/exp/FXE/201930/p900063/raw/r0844/RAW-R0844-AGIPD01-S00008.h5')  # DA -> AGIPD
<re.Match object; span=(0, 72), match='/gpfs/exfel/exp/FXE/201930/p900063/raw/r0844/RAW->

Types of changes

  • New feature (non-breaking change which adds functionality)

Reviewers

@calibration

Edited by Philipp Schmidt

Merge request reports