Allow Slurm jobs to fail, separate finalize job
Description
This is the second part of calibration/planning#34: allow the Slurm jobs running the notebooks to fail, and separate the 'finalize' step into its own job to ensure it still creates the report even if correction failed. It follows on from !510 (merged) (we may want to run with that one for a while before merging this, although I don't foresee any problems from combining them).
This also simplifies the code running the notebooks a bit, by separating the running of the notebooks from the conversion & report production.
How Has This Been Tested?
Run xfel-calibrate on Maxwell, verified that the report is still created in the output directory (/gpfs/exfel/data/scratch/kluyvert/agipd-calib-900201-203-sep-finalize2).
Relevant Documents (optional)
Types of changes
- Bug fix (non-breaking change which fixes an issue)
- Refactor (refactoring code with no functionality changes)
Checklist:
- My code follows the code style of this project.