[webservice] feat: Processing calibration logs and displaying errors/warnings on myMDC

Review changes
Download
Patches
Plain diff

Open [webservice] feat: Processing calibration logs and displaying errors/warnings on myMDC

feat/webservice_process_cal_logs into master

Overview 17
Commits 10
Pipelines 0
Changes 1

Open Karim Ahmed requested to merge feat/webservice_process_cal_logs into master 4 months ago

Overview 17
Commits 10
Pipelines 0
Changes 1

Description

This MR addresses this issue and is the next step after storing CAL warnings/errors to log in JSON format

Here process_request_finished is updated. CAL logs are processed after each execution is finished and we check for warnings and in cased of failed execution we check for errors. The log message sent to myMDC is formated in a way to show the messaged for each karabo_id and the number of jobs failed for each message.

WIP: The shown screenshots are for errors. I will add some user warnings for testing as we dont have yet any to show and test how the messages are displayed in case we have multiple messages for the same jobID

%%{init: {'theme': 'base', 'themeVariables': { 'fontSize': '10px', 'fontFamily': 'arial','nodeSpacing': 40, 'rankSpacing': 40}}}%%
graph TD
    GetExec["For each execution:<br/>Get karabo_id & status"] --> CheckExec{Execution<br/>Successful?}
    
    CheckExec -->|Yes| WarnOnly["Check warnings only<br/>warnings[karabo_id][msg] += job_id"]
    
    CheckExec -->|No| StatusCheck{"Check job status<br/>COMPLETED/FAILED/OTHER"}
    
    StatusCheck -->|FAILED| ErrorLogs["Check error logs<br/>errors[karabo_id][msg] += job_id<br/><i>or 'warning message for no error log found'</i>"]
    StatusCheck -->|OTHER| AddStatus["Add status message<br/>errors[karabo_id]['SLURM status'] += job_id"]
    
    ErrorLogs & AddStatus --> CheckWarns["Check warning logs<br/>warnings[karabo_id][msg] += job_id"]
    
    WarnOnly & CheckWarns --> Format["Format messages:<br/>{karabo_id: {<br/>  message: '16 jobs (id1-id16)',<br/>  status: '1 job (id17)'<br/>}}"]
    
    Format --> SendMDC["Send calibration feedback<br/>to myMDC"]

    classDef process fill:#dfd,stroke:#333,stroke-width:2px;
    classDef decision fill:#ffd,stroke:#333,stroke-width:2px;
    classDef data fill:#ddf,stroke:#333,stroke-width:2px;
    
    class GetExec,WarnOnly,ErrorLogs,AddStatus,CheckWarns process;
    class CheckExec,StatusCheck decision;
    class Format,SendMDC data;