Offline Calibration Reportservice

The Reportserivce is a service responsible for handling requests (manual or automatic triggers) for generating the DetectorCharacterization reports based on the requested configurations.

The Reportservice mainly consists of a service, clients and YAML configuration. The service keeps on listening to any ZMQ requests with a given configurations. Then based on these configurations, it produces slurm jobs (through xfel-calibrate command line) to generate *.png plots of calibration configurations over time.

Finally these generated plots are pushed on to DetectorCharacterization(DC) remote repository and displayed on the ReadTheDocs(RTD).

Configuration

It is important to know the machine name and the port, where the reportservice is running, for successful connection.

Starting the Service

The reportservice is a python script that can run through:

```bash
python reportservice.py
```

The available command line arguments are:

• --report-conf: The path for the main report configuration yaml file.
• --log-file: The path for the log file.
• --mode: The mode for running the service. Choices are sim[simulation], local and prod[production].
• --logging: The required logs to be written. Choices are INFO, DEBUG and Error.

Modes:

prod is the production mode working on the max-exfl016 as xcal user for generating the DC report through RTD and it should generate a very generalized DC report for the available detectors with information useful for most of the detector experts and users.

local is the mode used for generating figures locally without uploading the DC report on RTD or pushing figures to the git repository, rather generated figures are copied to the local repository and depending on the given report-fmt(report format) argument an html or a pdf is generated in doc/_build/ of the report service out folder (repo-local).

sim is a simulation mode, which is mostly used for debugging purposes and tool development without generating any reports locally or over RTD. This mode make sure not to do any git interactions and it only works on running the notebooks for generating figures in the out-folder without further work.

Report Configuration(report-conf):

report_conf.yaml is the configuration file, which contains all the required information for operating and connecting to the reportservice.

The global information holds the path to the DetectorCharacterization main tool for displaying the plots. Also, the information over the server (reportservice.py) and the time for automatic triggers for updating DC plots.

    Global
        git:
            repo-local: "/gpfs/exfel/data/scratch/<username>/DetectorCharacterization/"
            figures-remote: "http://<username>:<git-access-tocken>@git.xfel.eu/gitlab/detectors/DetectorCharacterization.git"
            server-port: "tcp:<host-name>:<port-address>"

        report_service:
            report-service:
            port: <port-address>
            bind-to: tcp://*
            job-db: ./reportservice_jobs.sqlite
            job-update-interval: 30
            job-timeout: 12000

The YAML configuration file can be modified with all the available parameters, responsible for generating the required plots and DC report on RTD.

    <instrument>:
        <detector>:
            det-type:

            start-date:
            nconstants:
            end-date:

            constants:
                - "Noise"
                - "Offset"

            modules:

            bias-voltages:

            mem-cells:

            out-folder:
            cal-db-timeout: 180000
            cal-db-interface: "<cal-db-host-port>"

Triggering the service

To use the service and generate a DC report corresponding to the report_conf.yaml. The service can be triggered through two processes:

Automatic Launch:

This is a similar script to reportservice.py that needs to be run using:

```bash
python automatic_run.py
```

• --config-file: The path for the configuration file* --log-file: The path for the log file.
• --logging: The required logs to be written. Choices are INFO, DEBUG and Error
• --log-file: The path for the log file.

Manual Launch:

This manual launch script is currently used for debugging purposes, only.

The available command line arguments are:

• --config-file: The path for the configuration file
• --instrument: A selected list of instruments to generate a report for. This instrument must be in the report_conf.yaml. The default for this argument is ['all]
• --overwrite-conf: A bool for indicating a new report configuration file(conf-file) should be sent instead of the default report_conf.yaml, which is used by report_service.py from the start.
• --log-file: The path for the log file.
• --report-fmt: The output DC report format. Choices are pdf or html
• --upload: A bool for uploading the figure to out-folder of the report service(repo-local) and generating a report. Default is False
• --logging: The required log mode to be used. Choices are INFO, DEBUG and Error