Make metadata directory name match report filename

Description

Each PDF report has an associated metadata directory containing details of the calibration, including the information needed to reproduce it. But we noticed that the name of the report and the name of the metadata directory are in different formats, with slightly different timestamps.

This takes the report name and reuses it for the metadata directory.

FXE_DET_LPD1M-1_correct_003338_r120_230512_152319.pdf
FXE_DET_LPD1M-1_correct_003338_r120_230512_152319/

I've also tightened the requirements for passing in --report-to a bit, to simplify the code; if passed, it's simply treated as a path to a file, not special casing names of existing directories or folder-less file names. The .pdf extension is still added if not given, though.

Closes #76.

How Has This Been Tested?

TBD

Types of changes

• Bug fix (non-breaking change which fixes an issue)
• Breaking change (fix or feature that would cause existing functionality to not work as expected) - more restrictive --report-to

Checklist:

• My code follows the code style of this project.

Reviewers

@schmidtp @ahmedk

added 1 commit

• d864c987 - Allow for xfel-calibrate with no --karabo-id (as in tests)

Compare with previous version

added 1 commit

• fe43ab03 - Fix --report-to in tests

Compare with previous version

changed the description

Thank you, handling the --report-to argument has been quite a mess for a while. Pending tests, L(!)GTM.

Code LGTM

resolved all threads

I've tested it on max-exfl017, creating metadata directories like:

• /gpfs/exfel/exp/CALLAB/202031/p900113/usr/Reports_test/r9981/SPB_DET_AGIPD1M-1_correct_900113_r9981_230703_172150
• /gpfs/exfel/exp/CALLAB/202031/p900113/usr/Reports_test/r9999/FXE_XAD_JF1M_correct_900113_r9999_230703_172155
• /gpfs/exfel/d/cal_tst/caldb_store/xfel/reports_test/CALLAB/MID_EXP_EPIX-1/dark/dark_900113_r9988_230703_172934

These do match the report filenames (minus the .pdf extension) as expected.

Karim asked me to work out when we keep fractional seconds in the timestamp and when we don't. It appears that when you request calibration through the webservice, it specifies a --report-to path with only second precision, and when you run xfel-calibrate directly with no --report-to, you get microseconds. I guess we should use the microseconds in the webservice as well, to better cope with any future issues causing two requests to be sent in quick succession.

added 1 commit

• c9f2f3d2 - Use fractional seconds in timestamps from webservice

Compare with previous version

added 1 commit

• 4bfd6f8f - Use _ instead of . to mark fractional seconds for report path

Compare with previous version

Folder names and report filenames now include the microseconds, as in:

• /gpfs/exfel/exp/CALLAB/202031/p900113/usr/Reports_test/r9998/FXE_XAD_JF1M_correct_900113_r9998_230703_182958_242803
• /gpfs/exfel/d/cal_tst/caldb_store/xfel/reports_test/CALLAB/MID_EXP_EPIX-1/dark/dark_900113_r9988_230703_183153_972118

I'll merge this tomorrow unless anyone wants to look at it further.

merged

mentioned in commit 10ba2381

changed milestone to %3.11.0

mentioned in merge request !876 (merged)