Send 'E' and 'IP' statuses to myMdC for correction jobs

Description

Luis tells me that proposals set to auto-calibrate migrated runs have a cron job which will launch calibration for any runs with calibration status 'NA' (not available). We've been setting this when calibration fails, which could lead to an infinite loop of calibration failing and restarting. I don't think we've seen this yet - we fixed a different infinite loop in !796 (merged) - but now that we have a separate 'E' for error status, we should use that instead.

Update: this now also uses the new 'IP' (In Progress) status for running jobs, instead of 'R'. The myMdC change was deployed on Saturday (2023-02-11).

How Has This Been Tested?

Deployed on max-exfl017, see below.

Types of changes

• Bug fix (non-breaking change which fixes an issue)

Checklist:

• My code follows the code style of this project.

Reviewers

@maial @schmidtp @danilevc

LGTM

Thank you.

Just for documentation, here is the list of values currently accepted for flag flg_cal_data_status

https://git.xfel.eu/ITDM/metadata_catalog/-/blob/develop/app/models/run.rb#L156

Please note that option IP (In Progress) was discussed last Thursday afternoon and deployed in production last Saturday (11 February 2023).

Thank you. Cheers,

This definitely looks like the right thing to do, LGTM.

It's not clear to me this will always set a proper status string, but that is something to overhaul anyway with more widespread errors and warnings.

mentioned in merge request !796 (merged)

added 1 commit

• c4850c8c - Replace 'R' status for correction with 'IP'

Compare with previous version

changed title from Replace setting 'NA' in myMdC with 'E' for error to Send 'E' and 'IP' statuses to myMdC for correction jobs

changed the description

I've also changed 'R' to 'IP' now. Only the job monitor will actually send 'IP' status, so the status won't change until the Slurm jobs start.

I've just tested this on max-exfl017. There are some oddities with how the new IP status is displayed in myMdC, but it appears to be working from our side:

In the run list (IP, E):

In the individual run view:

changed the description

@maial it looks like myMdC lets you start a new calibration while it's in progress (at least, the 'Start new calibration' option looks active in the dropdown. We should probably block that, like you already do for 'R' status.

Thank you, @kluyvert.

Let me take a look

Hi Thomas,

Both issues were fixed yesterday in the test system:

We will upgrade Production next Monday unless you are planning on deploying it earlier in production.

Thank you. Cheers,

Thanks Luis! I would assume we'll deploy on the next blue week, unless there are pressing issues we need this to fix.

Perfect Thomas.

I will deploy it anyway next Monday because our code is independent of yours.

Thank you!

Sounds good, thanks. I'm going to take the previous LGTMs and the testing I did to be sufficient confirmation that I can merge this.

merged

mentioned in commit 23a4eaf2