Fix/requirements into setuppy

Description

• Removes the requirements.txt file
• Move requirements into setup.py
• Reformat setup.py
• Update README to reflect no longer using requirements.txt
• Update CI to use new setup
  • Generates venv for tests each time to ensure runner local env is not used
  • Added --ignore=tests/legacy flag to pytest to ignore legacy test dir
  • Added --cov=cal_tools and --cov=xfel_calibrate to get coverage output for both modules
  • Added ssh key variable to CI, added .setup-ssh-euxfel-gitlab template to CI
• Change internal dependencies to use SSH instead of files (this is now possible as xsoft has a keypair on gitlab)

These changes make the installation of the package a bit more standard, removes the step of having to run two install commands (one for requirements.txt, and one for .), and let us use extras to define additional optional dependencies (e.g. test, dev, etc...).

How Has This Been Tested?

Tested locally by running pip install without extras, with test, with dev, and with test,dev, then ran the pytests to check that everything was still working correctly.

Also updated the CI definition to use the new setup style, so the tests also executed with the pipeline.

Relevant Documents (optional)

Types of changes

• Bug fix (non-breaking change which fixes an issue)

Checklist:

• My code follows the code style of this project.
• My change requires a change to the documentation.
• I have updated the documentation accordingly.

Reviewers

@danilevc @ahmedk @hammerd

Notes:

For our local dependencies the name of the package doesn't match the name of the repository, so I have to explicitly write out the name, and then use the git ssh url as a target, e.g. iCalibrationDB @ git+ssh://git@git.xfel.eu:10022/detectors/cal_db_interactive.git@2.0.1. @kluyvert does this look correct?

README.rst

@@ -25,7 +25,6 @@ Installation using python virtual environment - recommended
 4. ``source .venv/bin/activate`` - activate the virtual environment
 5. ``python3 -m pip install --upgrade pip`` - upgrade version of pip
 6. ``python3 -m pip install .`` - install the pycalibration package (add ``-e`` flag for editable development installation)
-7. ``pip install "git+ssh://git@git.xfel.eu:10022/karaboDevices/pyDetLib.git#egg=XFELDetectorAnalysis&subdirectory=lib"``
 
 Copy/paste script:
 
@@ -38,7 +37,6 @@ Copy/paste script:
   source .venv/bin/activate
   python3 -m pip install --upgrade pip
   python3 -m pip install .  # `-e` flag for editable install, e.g. `pip install -e .`
-  python3 -m pip install "git+ssh://git@git.xfel.eu:10022/karaboDevices/pyDetLib.git#egg=XFELDetectorAnalysis&subdirectory=lib/"
 
 
 Installation into user home directory
@@ -198,8 +196,8 @@ yourself first:
 
   salloc -p exfel/upex -t 01:00:00
 
-where `-p` gives the partition to use: exfel or upex and `-t` the duration the
-node should be allocated. Then `ssh` onto that node.
+where `-p` gives the partition to use: exfel **or** upex and `-t` the duration
+the node should be allocated. Then ``ssh`` onto that node.
 
 Then activate your environment as described above (or just continue if you are
 not using a venv).
@@ -226,15 +224,16 @@ Finally run the script:
   python3 calibrate.py --input /gpfs/exfel/exp/SPB/201701/p002012/raw/r0100 \
     --output ../../test_out --mem-cells 30 --detector AGIPD --sequences 0,1
 
-Here `--input` should point to a directory of `RAW` files for the detector you
-are calibrating. They will be output into the folder specified by `--output`,
-which will have the run number or the last folder in the hierarchy of the input
-appended. Additionally, you need to specify the number of `--mem-cells` used for
-the run, as well as the `--detector`. Finally, you can optionally specify to
-only process certain `--sequences` of files, matching the sequence numbers of
-the `RAW` input. These should be given as a comma-separated list.
+Here ``--input`` should point to a directory of ``RAW`` files for the detector
+you are calibrating. They will be output into the folder specified by
+``--output``, which will have the run number or the last folder in the hierarchy
+of the input appended. Additionally, you need to specify the number of
+``--mem-cells`` used for the run, as well as the ``--detector``. Finally, you
+can optionally specify to only process certain ``--sequences`` of files,
+matching the sequence numbers of the `RAW` input. These should be given as a
+comma-separated list.
 
-Finally, there is a `--no-relgain` option, which disables relative gain
+Finally, there is a ``--no-relgain`` option, which disables relative gain
 correction. This can be useful while we still further characterize the detectors
 to provide accurate relative gain correction constants.
 
@@ -247,23 +246,23 @@ Important information that doesn't really fit in as part of the readme.
 
 TODO: Place this into the docs? Also, improve docs (out of scope for PR !437)
 
-GitLab Access for xcaltst and xcal
-==================================
+GitLab Access for ``xcaltst`` and ``xcal``
+==========================================
 
-To make it easier to work with and deploy software via xcaltst/xcal, we have
-created an xcal account for gitlab with the following details:
+To make it easier to work with and deploy software via ``xcaltst``/``xcal``, we
+have created an xcal account for gitlab with the following details:
 
 - Full Name: ReadOnly Gitlab Calibration External
 - User ID: 423
-- Username: xcalgitlab
+- Username: ``xcalgitlab``
 - Password: ask Robert Rosca
 
 This account is intended to be used as a read only account which can be given
 access to certain repos to make it easier to clone them when using our
 functional accounts on Maxwell.
 
-The xcaltst account has an ed25519 keypair under ``~/.ssh/gitlab/``, the public
-key has been added to the xcalgitlab's approved SSH keys.
+The ``xcaltst`` account has an ed25519 keypair under ``~/.ssh/gitlab/``, the
+public key has been added to the ``xcalgitlab``'s approved SSH keys.
 
 Additionally this block has been added to ``~/.ssh/config``: