changelog.md

Release Notes

3.15.0

[AGIPD][Correct] New feature for lit-pixel counter [AGIPD][Correct] Refactor write_file on AgipdCorrections to create indexes using the DataFile [AGIPD][PC] Fixing plotting in summary notebook [AGIPD][LPD] Add DataFile.create_legacy_source [LPD][Correct] Write corrected data as new source and link to old source name [Jungfrau][Dark] use CalibrationData to retrieve prior CCVs [Jungfrau][Correct] Test ROI for JF1M [Jungfrau][Correct] use new correct data source and link to old data source [Jungfrau] roi-definitions to update_config.py script [PNCCD][Correct] New corrected data source and link to legacy source [Epix100][Correct] New corrected data source and a link to old data source [Gotthard2]: Avoid getting wrong data sources or including control devices' sources. [xfel-calibrate]: expose mincpus argument as --slurm-mincpus [Fix] Pin the pillow version [Test] Configure and Include new test runs [Test] Swap order of dicts for DeepDiff [Webservice] Remove extra slash from reports-folder config [Webservice] Use MUNGE credential with update_conf requests [CalCatAPI] port markdown tables of found constants [CalCatAPI] Convert digits and boolean to float and the rest to string Make constant tables in both markdown & latex format

3.14.3

Update report creation to use texlive/2022 fix: Update documentations after EL9 update

3.14.2

[AGIPD][Correct][CS] Fix badpixelsCS before we have multiple CCVss to have same BadPixels shapes. [AGIPD] Modernize use of extra_data to get AGIPD control data [AGIPD][Correct] use np.nanmin and np.nanmax to avoid nans when modules are missing [AGIPD][Correct] bug when plotting gain data of only 0 after it's info was discarded by DAQ [Shimadzu][Correct] fix for less expected images Remove unnecessary 'module load' command from jobs to be repeated Remove reference to anaconda/3 module flake8 repository has moved from gitlab to github

3.14.1

[AGIPD][CORRECT][DARK] Break wrong assumption for availability for 1st module. [AGIPD][CS] new current source calibration constant. [AGIPD][CS] Correction using Current source slopes. [Shimadzu][CORRECT] Fix name of index group in output files. [JUNGFRAU][FF] Feat: new notebook for producing gain constants. Feat/ci new runners python311. Create an unreadable temp directory for testing PermissionError.

3.14.0

• [shimadzu] Add support ofr PCA-based dynamic flat-field corrections of Shimadzu HPVX2 data.
• [AGIPD] Fix PC processing to allow for a single module.
• [AGIPD] Remove default value for caldb_root.
• Add staging version of new CalCat API to retrieve calibration data (to be superseded by extra.calibration).
• Add API to create calibration constant data files and inject directly into CalCat.
• Configure number of required characterization runs from CalCat based on operation mode.

3.13.0

• [AGIPD][PC] Fixing ValueError.
• [Gotthard2][Dark] Sort dark runs.
• [ePix100][Correct] Correcting one train for epix100 and storing a list of one pulseId.
• [Doc] Change log update with latest releases.
• [Gotthard2][Correct] Add idx to da_to_pdu to use in picking correct data_source.
• [LPD][Dark] Sort Dark runs by Gain.
• [Webservice] Use AW status on myMdC for warnings that prevent launching correction.
• [Webservice] Move list of three gain/run detectors into webservice config yaml.
• [Webservice] add environment bin/ to PATH if not already there.
• Convert request time to local timezone in report.
• Don't keep DEBUG level logging from requests_oauthlib.
• Avoid using run_metadata() by default, to not fail for EXDF-v0.5 files.

3.12.6

• [Jungfrau][Correct] Avoid NAN disturbance for correction plots scale
• [Timepix] Updating Timepix3 Calibration Pipeline for compatibility with new Karabo Implementation + some general improvements
• [Timepix] Allow pipelines to be part of nodes in DataFile API
• [Timepix] Throw pasha at it
• [Timepix] Store labels assigned to pixels

3.12.5

• SPARTA AGIPD Single Module integration

3.12.4

• [Gotthard2][Correct] Store mask data properly for gotthard2 25um
• Make the run type regex for skipping JF darks less aggressive

3.12.3

• [webservice] Update Kafka config to use EuXFEL broker

3.12.2

• [GH2] Temporary workaround for processing HIREX detector's dark runs using the correct data sources
• [GH2][Correct] Show number of available trains and raise ValueError for runs with no trains
• [GH2][Correct] New paramater to reverse second GH2 25um module
• [Jungfrau][Correct] Fix not returning after finding a dark fixed gain constants
• [Jungfrau][Dark][Correct] Improve Jungfrau plots
• [Webservice] Add new detector HIREX to 3 dark runs detectors

3.12.1

• [AGIPD][PC] Summary notebook for PC processing
• [GH2][Correct][Dark] Break assumptions on receiver names
• [Webservice] Create correction reports in MyMdC when jobs have finished
• [Webservice] Use leading zeros in usr/Reports folders
• Make serve_overview resistant against missing karabo-da in data-mapping
• Add links to DESY grafana for running jobs in serve_overview

3.12.0

• [AGIPD][Correct] Bug for using drop with a list in AGIPD CORR NB
• [AGIPD][Correct] Try to simplify & speed up file reading code
• [GH2][Correct][DARK] Feat/add support for gh2 25um
• [Jungfrau][Correct] Update rows A1256 LUT after rotation
• [Jungfrau][Dark] Account for dark runs not taken by the MDL device
• [JUNGFRAU][Dark] Fix bad pixel constant from darks taken in burst mode
• [REMI] Add and enhance plots, more documentation, clean-up and some deprecation
• [TEST] Update AGIPD tests with xray-gain, add DSSC, ePix100 tests and update reference folder path
• [TEST] clearer comparison of HDF5 files
• DataFile: Don't write creationDate & updateDate unless specified

3.11.5

• Update CalParrot==0.3 and EXtra-data==1.15.1 dependencies
• [DSSC][Dark][Correct] No longer restrict memory cells to a multiple of 100s and add lower deviation for memory cells parameter conditions.
• [Webservice] Catch errors on failure to launch dark processing
• Add script to update dark run status in myMdC
• [Epix100][Correct] Calcat error when no gain is retrieved
• [REMI] Disable trailing trigger by default

3.11.4

• [Jungfrau][Correct] Force fixed gain for JF data in burst mode

• [Jungfrau][Correct] Force replacement for gain value in Jungfrau correction

• [DSSC] Allow 900 memory cells for DSSC darks to workaround appearance of cell 810

• [Jungfrau][Dark] Reflect WRONG_GAIN_VALUE over a pixel in all gain for badpixels map

• [AGIPD][Dark] Add timings

• [Jungfrau][Correct] New A1256 JF Strixel

• [webservice] Add JUNGF and PEP 8 on the line

3.11.3

• [AGIPD][LPD][DARK] Show table for bad pixels bitmaps

• [AGIPD][CORRECT] Process all AGIPD trains if the PPU device is missing or if no trigger

• [AGIPD][DARK] Sort dark runs

• [AGIPD][DARK] Improvements for reading conditions by creating a new data class for multiple runs

• [AGIPD][FF] Fixing FF summary performance plots

• [Jungfrau] [Correct] Add thresholding for ROI projections

• [Jungfrau][Correct][Dark] Fix manual edit for operating conditions

• [Jungfrau][DARK] Validate and reorder dark runs before processing

• [EPIX][FF] ePixFF characterization

• [REMI] Add support for virtual trailing trigger

• Fix manually submitting the confirmation

• move some logs to DEBUG and extend the report sleep

• Expose --blc-stripes to update_config.py and fix old parameter names

3.11.2

• Operational release for SPB to support configurable rounding thresholds.

3.11.1

• [AGIPD][CORRECT] Use calcat_interface and remove precorrection notebook
• [EPIX100] Feat: Compliance with update to receiver device
• [REMI] Various fixes and improvements for quad DLDs
• [REMI] Fix missing re-allocation of trigger array with neither FEL nor PPL
• [Tests] Fix: Accept uppercase calibration type
• [Test] Find difference by default
• Clearer error when xfel-calibrate would run no jobs
• Detect cycle automatically in update_config script
• Fix link to CalCat

3.11.0

• [AGIPD][Correct] Handle selecting multiple trains per PPU trigger
• [AGIPD][Dark] Fix: Skip corrupted frame from dark processing
• [LPD1M] Automatically decide whether to inject & use memory cell order
• [LPD1M][Dark] Use EXtra-data to create darks from >1 sequence file
• [LPD1M][Correct] Use parameter names instead of IDs to find constants
• [LPD1M][Correct] Using CALCAT interface
• [LPD1M][Correct] Fix: Constant type conversion
• [LPD1M][Correct] Use the fragment file and remove the precorrection notebook
• [LPD-Mini] Rework cell order condition to match LPD-1M again
• [JUNGFRAU][pnCCD][ePix100] Feat: new method to display CCV metadata in reports
• [JUNGFRAU][CORRECT] Add fragment file and remove precorrection notebook
• [EPIX][DARK] Mark dead pixels as Bad Pixels
• [EPIX][CORR] Optimize histograms and plots
• [GH2][Correct] Move false warning and disable gain correction as printed.
• [GH2][Correct] Remove the precorrection notebook and add fragment
• [TIMEPIX] Add select parameters to update_config
• [TIMEPIX] Fix types of notebook arguments
• [xfel-calibrate] Fix: Break the line properly into latex when the next line starts with _
• [Webservice] Don't mark jobs as finished just because they disappear from squeue output
• [Webservice] Use status AW in myMdC if correction failed for some detectors in a run
• Add a pytest to run a dict of CALLAB test runs before releases
• Look up CCVs using parameter_name in place of parameter_id
• Replace max-exfl016 and max-exfl017 into max-exfl-cal001 and max-exfl-cal002, respectively.
• Make metadata directory name match report filename
• Add reorder_axes function

3.10.3

• [LPD][Correct] Harden against empty sequencee sets with train-on-demand
• [JF][correct] Add missing gain mode parameter
• [Timepix3] Add centroiding notebook

3.10.2

• [PNCCD][CORRECT] Fix: Skip error for missing gain

• [PNCCD][CORRECT] Fix: Hack to wrong ctrl bias voltage values p002857

• [LPD][Correct] Fix axis order for LPD-1M RelativeGain constant

• [LPDMini][Dark] Add only number of available data trains into data_samples

• [LPDMini][Dark] Fix first notebook cell to execute CL through the webservice

• [LPDMini] Feat: Inject gain constants notebook

• [Jungfrau] Workaround for SPB_CFEL_JF1M as the modules start with 09 not 01

• Fix update_config to work with non-AGIPD and add REMI

3.10.1

• [JUNGFRAU][CORRECT] Using calcat interface

• [JUNGFRAU][CORRECT][DARK] Extend accepted detectors based on substrings of karabo_id

• [LPD Mini][CORRECT][DARK] Initial work on LPD Mini notebooks

• [PNCCD][CORRECT] Avoid raising a CalCat error while retrieving metadata for missing gain constant from DB

3.10.0

• [ePix100][Correct] Avoid including histogram calculator with empty array

• [ePix100][Correct] Remove pre notebook

• [pnCCD][Correct] Using calcat interface

• [pnCCD][Correct] Record fragment file and remove pre correction notebook

• [ePix100][pnCCD][Correct] Display creation time for retrieved constants

• [AGIPD][Correct] error out only if all sources are empty

• [AGIPD][Correct] exit notebook if no correction files are found for the selected sequence

• [DSSC][DARK] Group all slow data to the same aggregator

• [JUNGFRAU][Correct] Use DataCollection.from_paths for reading JF CORR files for plots

• Support for saving metadata fragments & merging into calibration_metadata.yml

• [REMI] Save pulse amplitudes during discrimination

• [REMI] Make plots robust against no edges or no hits in data

• [Webservice] Add script to check run in webservice DB

• [Webservice] Fix database lock timeouts

3.9.2

• [AGIPD] Adding back overwrite parameter
• [AGIPD] Another hotfix for experimenting DAQ filtering at SPB

3.9.1

• Hotfix version for 3.9.0
• [AGIPD] LitFrameFinder bug fix for experimenting DAQ filtering at SPB

3.9.0

• [GH2][CORRECT]CALCAT interface.

• [ePix100][CORRECT]CALCAT interface.

• [pnCCD][CORRECT]Use DataFile to store aligned corrected data.

• [GH2][CORRECT]Use DataFile to store aligned corrected data.

• [ePix100][CORRECT]Use DataFile to store aligned corrected data.

• [AGIPD][SlopesFF][CORRECT]Add the deviation for all possible memory cells and enable correction using these FF constants with more memory cells.

  • !806
  • !613

• [AGIPD][CORRECT]Fix checking ccv_variant condition for AGIPD.

• [DSSC][Dark]Remove unused instrument parameter from DSSC dark notebook

• Update nbparameterise to 0.6.

• Update EXtra-redu to version 0.0.7

3.8.1

• Hotfix version for 3.8.0 with
• update extra_redu version

3.8.0

---

• [AGIPD][CORRECT] Clean up before AGIPD calcalt_interface changes
• [AGIPD][FF] Styling modification for both FF notebooks:1st nb cell, and removing unneeded imports
• [LPD] uses memory cell order as a condition for constants
• [LPD] [Correct] Don't pass default snapshot_at=None to calibration_client
• [LPD][JF][DataFile] Add parallel dataset compression
• [JF] Replace strixel cython code by NumPy implementation
• [EPIX100][CORRECT] Include gain and charge sharing to corrected data.
• New CALCAT interface
• Store CalCat requests/responses for reproducibility
• Update to calibration_client 11.2.0

3.7.6

• [AGIPD][CORRECT] Fix common mode correction: array reshaping and hardcoded 256 trains per file.

• [AGIPD][LFF] Improve lit frame selection

• [AGIPD][CORRECT] Fix common mode correction

  • Array reshaping.
  • Hardcoded 256 trains per file.

• [AGIPD][LFF] Improve lit frame selection:

  • fix the misalignment of selection by trains.
  • introduce super pattern selection.
  • guess missed patterns for patterns that repeat with constant step.
  • shows patterns that repeat with a constant step as one line in the report.

3.7.5

16-11-2022

• JUNGFRAU Fix index location for ROI instrument output !765

3.7.4

15-11-2022

• JUNGFRAU Only emit a warning if all sequences of a job are empty
• JUNGFRAU Fix storing multiple ROIs for a single module
• JUNGFRAU Replicate a raw dataset in processed data for legacy analysis code

3.7.3

10-11-2022

• DataFile Add support for older INDEX and METADATA versions.
• LPD Fix histogram plotting if selected cell's data is identical for trains.
• JUNGFRAU Avoid unclear error messages when the run has no trains.
• AGIPD Change notebook-default setting of max-task-per-worker to 1.
• REMI Allow pulse separation in REMI_DLD reconstruction to ignore FEL pulses.
• webservice Fix timestamps in the overview page.
• webservice Check run type more directly in myMdC response.

3.7.2

28-10-2022

• JUNGFRAU Add support for strixel sensors
• JUNGFRAU Shorter summary notebook for darks by default
• JUNGFRAU Always align corrected data properly
• JUNGFRAU Harden against 0 adc data in dark characterization
• REMI Add support for quad nodes and various pulse separation improvements
• AGIPD Allow manual photon energy input for photonization and add consistency plots
• Grant priority partitions for set-up time

3.7.1

14-10-2022

• EPIX100 Add Common mode corrections to dark characterization
• EPIX100 Use a single value for temperature
• JUNGFRAU Fix bad pixel masking
• JUNGFRAU Limit number of plotted trains to 500 during corrections
• AGIPD Don't fail when there are no trains in sequence file
• REMI Use DataFile API and support asymmetric pump-probe patterns
• Skip corrections for runs which match specific types

3.7.0

07-09-2022

• JUNGFRAU Mask double sized pixels !689
• LPD Storing retrieved calibration constants in calibration_metadata.yml.
• Storing the running logs in the report folder path instead of a [temp] folder wherever the processing ran from.
• Selecting partitions based on beam time dates. !725
• Creation time is not read from myMDC anymore.

3.6.4

20-08-2022

• AGIPD Significant improvements to AGIPG LitFrameFinder implementation, including offline support.
• AGIPD Fix misalignment of data and gain/mask when common mode is used with manual pulse slicing.

3.6.3

23-08-2022

• AGIPD litframe finder configs in update_config !723

3.6.2

15-08-2022

• EPIX100 - JUNGFRAU - pnCCD Retrieve constants precorrection notebooks.
• AGIPD skip sanitization !654
• AGIPD New PC notebook !604
• JUNGFRAU Allow selecting ROIs to save 1D projections !699

3.6.1

26-07-2022

• GH2 Correction and Dark notebooks. !658, !684, !707
• pin [xarray] to 2022.3.0 !706

3.6.0

20-07-2022

• LPD - DARK Save bad pixel dark data as uint32, not float64. !698
• LPD - CORRECT Add option to disable Oauth when using CalCat proxy. !681
• AGIPD - LPD - DSSC - DARK Avoid printing errors for expected missing constant files. !664
• TESTS Refactor and split [AGIPDCtrl] methods (old and new RAW data versions.) and add unit tests !651, !646
• webservice Monitor SLURM jobs in a separate process. !668
• webservice Restructure the SLURM database to give more meaningful success/failure information. !683
• XFEL-CALIBRATE Recreate environments on demand for xfel-calibrate-repeat. !660
• Upgrade the Jupyter packages to the latest version. !697 (diffs)
• Update Extra-data to 1.12: !693
  • [EXtra-data] Disable [_use_voview] to be able to use [.files] or avoid using [.files] !682, !688

3.5.5

15-06-2022

• AGIPD - CORRECT Expose max tasks per pool worker.

3.5.4

13-06-2022

• AGIPD Convert bias_voltage parameter condition to integer in cal_tools.
• LPD Fix correcting a single pulse.
• LPD VCXI require 4 modules.

3.5.3

19-05-2022

• LPD - CORRECT Optionally create virtual CXI files
• LPD - CORRECT Expose max-nodes parameter
• AGIPD Replace gain_choose_int by fused types
• Fix missing install of restful_config.yaml
• Fix use of xfel-calibrate --skip-report

3.5.2

16.05.2022

• LPD - CORRECT New correction notebook for LPD
• New [files] module to write European XFEL HDF5 corrected data files.

3.5.1

05-04-2022

• Calibration Constant version's new [Variant] file attribute. To indicate method of handling the constant post retrieval. !619
• Epix100 dark Badpixels Map. !637
• skip-plots flag to finish correction before plotting. !635
• First trainId's timestamp as RAW data creation_time, if there is myMDC connection. !630
• AGIPD correction can correct one cellId without plotting errors. !642
• Fixed mode relative gain constants in Jungfrau can be retrieved. !639
• Only instrument source is selected to check number of trains to dark process. !636
• AGIPD trains for dark processing is selected for each module individually. !653
• Produce report after trying to correct AGIPD run with no images. !641
• AGIPD's bias voltage for AGIPD1M is read from slow data. !647
• Removed psutil dependency. !653
• Update Pasha to 0.1.1 !638

3.5.0

01-03-2022

• Updating Correction and dark notebooks for JUNGFRAU: !518
• Updating Correction and dark notebooks for AGIPD: !535
• Updating Correction and dark notebooks for PnCCD: !559
• Updating Correction and dark notebooks for ePix100: !500
  • EXtra-data is integrated to read files in pycalibration for AGIPD, JUNGFRAU, ePix100, and PnCCD. Dark and Correction notebooks.
  • Pasha is now used for processing data for JUNGFRAU, ePix100 and PnCCD.
  • pyDetLib correction functions were removed (except for common-mode correction).
  • [db-module] is useless now for JUNGFRAU, ePix100, and PnCCD. Some parameters were updated in dark and correction notebooks for the mentioned detectors.
• gain_mode and burst mode are now available for JUNGFRAU. !591
• JUNGFRAU has now a new badpixel value, WRONG_GAIN_VALUE. !591
• Pass through available for testing in-progress ORCA service. https://git.xfel.eu/calibration/pycalibration/-/merge_requests?scope=all&state=merged&search=orca
• Non-calibrated RAW h5files are no longer copied.
• High priority partitions ([upex-high and upex-middle]) are used for runs from ACTIVE and READY proposals, only. !628
• Supporting to disable LPD Correction through the webservice. !629
• Compatibility for old DAQ files for REMI is added. !607
• server-overview refactors. !593 !589
• AGIPD correction notebook support AgipdLitFrameFinder device. !603
• Parsing code arguments in xfel-calibrate is refactored. !575
• skip-plots option for AGIPD. !581
• Native implementation for transposition of constants AGIPD. !580
• Trains for AGIPD can be selected for correction. !574
• Skip report flag in xfel-calibrate. !579
• Fix ReadTheDocs. !448
• Fix error reporting for re-injecting the same CCV. !627
• Fix AGIPD for legacy runs without gain_mode. !617 !626
• Pinning markupsafe version 2.0.1 !631
• Pinning psutil 5.9.0 !535
• Updating Extra-data to 1.9.1 !535
• Updating h5py to 3.5.0 !602

3.4.3

20-10-2021

• Update pyDetLib tag.
• Add explicit dependencies on matplotlib, scipy.
• Remove outdated matplotlib rcParams setting.
• Update EXtra-geom to 1.6.
• Remove cluster_profile parameter from notebooks which don't use it.
• Fix checking availability for the concurrency parameter.
• Fix launching work directly (not via Slurm).
• Fix sphinx-rep temp folder recreation, if sphinx-rep already existed.
• Fix missing string conversion for slurm-scheduling argument.
• Fix title reports for multiple detectors per run folder.
• Append to .out files for preemptable finalize job.
• AGIPD CORRECT Reuse previously found constants.
• AGIPD Fix missing memory cell index in SlopesPC constant sanitization.
• AGIPD Only use bad pixels from darks in agipdutils.baseline_correct_via_stripes.
• AGIPD DARK Use function to get list of karabo_da from run for making Slurm jobs.
• EPIX100 - CORRECT Set absolute_gain to false if relative gain was not retrieved.
• JUNGFRAU Fix running for multiple modules and flip logic for do_relative_gain.
• JUNGFRAU Style changes for Dark and Correct notebooks.
• REMI Add notebook to reconstruct detector hits from raw data.
• webservice Check run migration status using MyMDC.
• Resolve "Skip ZMQ tests if zmq connection for calibration DB not available".
• Reproducibility, step 1.

3.4.2

17-09-2021

• Remove driver=core from all notebook
• webservice Make use of Dynaconf for managing secrets.
• webservice Make use of dedicated slurm partitions.
• webservice Handle missing migration information (missing user.status fattr).
• webservice Implement, raise, and catch, migration errors to send mdc messages.
• webservice Simplify handling of user notebook paths.
• webservice Update princess to 0.4 (use Unix sockets).
• webservice Update MyMDC with begin and end times.
• webservice create output folder before copying slow data.
• AGIPD CORRECT read acq_rate from slow data.
• AGIPD - CORRECT Set default memory cells to 352.
• AGIPD CORRECT Set maximum pulses to correct based on file content.
• AGIPD - FF Correctly label legends in figures.
• AGIPD - FF Add HIBEF AGIPD500K and fix some issue with retrieval of conditions.
• JUNGFRAU Add Gain setting to Jungfrau notebooks.
• JUNGFRAU Fix max gain plot in LPD correct notebook
• JUNGFRAU DARK Clearer error message for Jungfrau Dark notebooks no suitable files are found
• LPD CORRECT Fix max gain plot.
• EPIX100 CORRECT Solve conflict between gain correction and clustering

3.4.1

16-07-2021

• Update h5py to 3.3
• Stop execution on notebook errors
• AGIPD Add integration time as operating condition to all notebooks
• webservice Add blocklist pattern when copying untouched files in webservice.
• webservice Expose dark configurations in update_config.py
• Fix MetadataClient.get_proposal_runs arguments call.
• Fix Use snapshot for injecting constants for old PDU mappings
• Fix the old time-summary (creation time for retrieved constants)
• Update documentation notes on venv installation
• Ignore all .so files in gitignore

3.4.0

28-06-2021

• Update to Python 3.8.
• Bump numpy to 1.20.3 and remove fabio.
• remove PyQT dependency.
• Disable dark requests from serve overview.
• Update report upload parameter key.
• Override locale to always use UTF-8.
• Assorted cleanup of xfel-calibrate.
• Fix pre-commit.
• Use argparse only if name is main, call main with args dict.
• webservice Use full hostname for webservice overview.
• webservice Show clearer messages when running webservice in sim mode.
• webservice Fix filename lineno and typos in webservice logs.
• webservice Fix creating an extra run folder in run output folder.
• AGIPD Parallelize gain/mask compression for writing corrected AGIPD files.
• AGIPD - DARK Fix processing empty sequence files.
• AGIPD - PC - FF Update notebooks with new CALCAT mapping.
• AGIPD - JUNGFRAU Use all available sequences for processing darks for AGIPD and Jungfrau.
• AGIPD - LPD - DSSC Fix retrieve old constants for comparison for modular detectors.
• LPD Fix data paths in LPD notebook.
• REMI Fix user notebook path for REMI correct notebook provisionally.
• EPIX100 - CORRECT Add Common mode correction.
• Fix plotting-related warnings.
• Test update config.
• Test get_from_db and send_to_db.