Tags

LPD (at FXE) had same issues with darks processing as DSSC (at SQS). Dropping "core" driver from notebook fixed it in that case, too.

Dropping the "core" driver in DSSC darks notebok seemed to resolve some issues (see https://in.xfel.eu/redmine/issues/102691)

Fix missing integration_time parameter in AGIPD dark notebook
Fix typo in AGIPD retrieve constants notebook when reading integration time parameter

Fix/agipd integration time pc hed
Fix MetadataClient.get_proposal_runs call arguments
Add blocklist pattern when copying untouched files in webservice
Add integration time as AGIPD operation condition to all notebooks
FEAT / Expose dark configurations in update_config.py
Update h5py to 3.3
Fix / Use snapshot for injecting constants for old pdu connection
Stop execution on notebook errors
Update notes on venv installation
Ignore all .so files in gitignore
Fix the old time-summary (creation time for retrieved constants)

Update to Python 3.8
    Update report upload parameter key
    Try to make pre-commit pass
    [AGIPD][DARK] Fix / Avoid processing empty sequence files
    Bump numpy to 1.20.3 and remove fabio
    Fix/filename lineno webservice logs
    Override locale to always use UTF-8
    Fix plotting-related warnings
    Fix user notebook path for REMI correct notebook provisionally
    Fix data paths in LPD notebook
    Test update config
    Test get_from_db and send_to_db
    Fix typo in log message
    [AGIPD][JF]Fix: Use all available sequences for processing darks
    add common mode correction to epix correction notebook
    Parallelise gain/mask compression for writing corrected AGIPD files
    remove PyQT dependency
    Disable dark requests from serve overview
    Remove line that creates a run folder within the run output folder
    Show clearer messages when running webservice in sim mode
    Use full hostname for webservice overview
    Use argparse only if name is main, call main with args dict
    Assorted cleanup of xfel-calibrate
    [AGIPD][DARK] Correctly retrieve old constants for comparison
    [AGIPD][PC][FF] FIX/Update pc and ff notebooks with new calcat mapping

Fix broken plots in summary notebook for AGIPD dark characterization. This is merged into master, this tag is for hotfix deployment.
https://git.xfel.eu/gitlab/calibration/planning/issues/56

- Major AGIPD dark speedup
- Support for SQS-REMI detectors
- Updates for relative gain of ePix100
- First set of tests added for xfel-calibrate
- Extend parallelization options for AGIPD corrections
- Misc bugfixes

Add missing sphinx dependency to setup.py
[AGIPD][CORRECT] disable rel_gain with fixed gain and remove unused parameter
Make max_nodes a parameter for AGIPD correction notebook
[AGIPD][CORRECT] Correctly set array of ones when no PC data available
Add --slurm-partition CLI parameter to xfel-calibrate
Fix key error in  modules_mappings

As this is partially hotfixes, we did a subset of the usual tests!

### Calibration Services

#### Dark Reports

[x] Success  
[] Failed  
**Comments**:  

#### Correction Reports

[x] Success  
[] Failed  
**Comments**:

#### MyMDC Correct Request

[x] Success  
[] Failed  
**Comments**:  

#### Kafka notifications

[x] Success  
[] Failed  
**Comments**:
```{'event': 'correction_complete', 'proposal': '900113', 'run': '9993', 'detector': 'fastccd', 'success': True}

```

#### MyMDC Dark Request

##### AGIPD500K@HED - run 9985

[x] Success  
[] Failed  
**Comments**:  

| Procedure                             | Expected Results      | Pass/Fail     | Actual Result/Comments  |
|-------------------------------        |------------------     |-----------    |------------------------ |
| Submit job from MyMDC                 | Requested status      | Pass          |                         |
| Submit job to slurm                   | In Progress status    | Pass          |                         |
| Watchdog updates job state            | Elapsed Time Updating | Pass          |                         |
| All slurm job finish                  | Finished status       | Pass          |                         |

### Calibration Notebook Dark/Correct Actions

#### Correct


##### AGIPD500K@HED - run 9985

[x] Success  
[] Failed  

##### EPIX10K@HED - run 9986

[x] Success  
[] Failed  

##### EPIX100@MID -run 9988

[x] Success  
[] Failed  

##### fastCCD@SCS - run 9993

[x] Success  
[] Failed  

#### Dark

##### AGIPD1M@SPB - runs: [9992, 9991, 9990]

[x] Success  
[] Failed  

##### AGIPD500K@HED - runs: [9985, 9984, 9983]

[x] Success  
[] Failed  


##### DSSC@SCS - runs: [9987]

[x] Success  
[] Failed

this is the same as 3.2.4 but with the branch hotfix/key_error_modules_mapping on top of 3.2.3

fix for key error `modules_mapping`