Simplify how batch jobs find Python tools

The idea here is that all the Python code which runs in a calibration job (like nbconvert, ipcluster & Sphinx) should be the versions in the same Python environment used to launch xfel-calibrate. This is achieved by launching tools with path/to/python -m module instead of scripts on $PATH. This is a common pattern we use to run Python modules in a specified Python installation.

The motivation is to have better control over what's going on - we don't want to silently run another version of one of these tools which happens to be found on PATH. This also makes it easier to install pycalibration in an environment (#26).

Closes #26 .

cc @kamile @ahmedk

resolved all discussions

added 1 commit

• 4660ac5f - Specify texlive version for module load

Compare with previous version

added 15 commits

• 4660ac5f...485a9a45 - 8 commits from branch master
• 910f9d79 - Rely on path/to/python -m for IPython, ipyparallel & nbconvert
• 37786afb - Run all Python subprocesses with the same Python
• b2c572eb - Fix comment about default Python
• a435c457 - Inline remaining activate script
• 5b0304f0 - Finalize script in Python
• 1cd9ae99 - Remove erroneous "
• 40ca67c0 - Specify texlive version for module load

Compare with previous version

Rebased on master to resolve some merge conflicts.

I have tested this by running the AGIPD correction pipeline, and it's working in so far as output data and the PDF report are still produced.

LGTM. Works the charm.

Thanks Cyril!

@ahmedk I'll leave this for now so you can have a look. If you're happy for me to merge it based on Cyril's review, let me know.

Thank you a lot. yes, I will review it today as well. Sorry, I didn't have the time to do it before.

No problem, just wanted to make sure we're on the same page about reviewing it.

yes, and the other PR as well.

LGTM!

Thanks!

merged

mentioned in commit 039f3933

mentioned in merge request !362 (merged)