Skip to content

feat(xfel-calibrate): expose mincpus argument as --slurm-mincpus

Karim Ahmed requested to merge feat/expose_slurm_mincpus_arg into master

Description

It was reported by @rovensky that there are inconsistencies in the processing time for the AGIPD FF notebooks that differ in terms of hours.

Checking the associated nodes for different processings with the same configuration leads to the conclusion that the node's available processors are the reason for this difference.

I have exposed the --mincpus sbatch argument in the sameway it is already done for --mem and --nice arguments.

--mincpus= Specify a minimum number of logical cpus/processors per node.

How Has This Been Tested?

Relevant Documents (optional)

Two AGIPD FF processing with the same configurations.

Both differ drastically in terms of performance. You can see each table of Runtime Summary along with the corresponding nodes and there number of CPUs.

  • First processing: /gpfs/exfel/exp/SPB/202331/p900376/usr/FF/GAIN_MAPS/352cells1.1MHz_gs0_20clk/FF_SPB_DET_AGIPD1M1_mc352_11mhz_gs0_900376_r8.pdf
AGIPDFF_1_runtime_summary AGIPDFF_1_nodecpuinfo
  • Second processing: /gpfs/exfel/exp/SPB/202331/p900376/usr/FF/GAIN_MAPS/352cells1.1MHz_gs0_20clk/FF_SPB_DET_AGIPD1M1_mc352_11mhz_gs0_900376_r8_1.pdf
AGIPDFF_2_runtime_summary AGIPDFF_2_nodecpuinfo

Types of changes

  • New feature (non-breaking change which adds functionality)

Checklist:

Reviewers

@kluyvert @schmidtp

Edited by Karim Ahmed

Merge request reports