Make max_nodes a parameter for AGIPD correction notebook

As requested for upcoming Covid beamtimes, this should allow controlling how many nodes an AGIPD correction job can use, by passing --max-nodes 16 at the command line or adding max-nodes: 16 in the proposal YAML config.

I haven't tested this; we don't have any long runs in CALLAB where setting a higher value would make a difference.

LGTM.

This can be tested locally with any proposal runs and have the output files on scratch, no need to test it with CALLAB.

Yes, good point; I've been tweaking the webservice so much lately that I'd forgotten about running it from the command line.

I don't suppose you happen to know of a long enough run to test with? I'll go hunting for one.

http://max-exfl016:8008//gpfs/exfel/exp/SPB/202130/p900201/usr/Reports/r203/SPB_DET_AGIPD1M-1_correct_900201_r203_210328_212700.pdf this is a report I found on the webservice with 23 sequences

I have tested the job submission. Going beyond nseq // 2 jobs requires setting --sequences-per-node 1 as well, but with that I was able to split one run into 24 jobs (--max-nodes 24).

All of my jobs then hung for an hour or so, seeming to get stuck at the point where they would write output data. So far, I haven't been able to work out what caused this, but it seems to have resolved itself when I cancelled & relaunched the jobs, so I'm crossing my fingers that it was one of those random GPFS issues.

mentioned in merge request !469 (merged)

Merging on the strength of @ahmedk's LGTM before I tested - I haven't made any changes since then.

merged

mentioned in commit 2556951b