Fix/requirements into setuppy

Description

• Removes the requirements.txt file
• Move requirements into setup.py
• Reformat setup.py
• Update README to reflect no longer using requirements.txt
• Update CI to use new setup
  • Generates venv for tests each time to ensure runner local env is not used
  • Added --ignore=tests/legacy flag to pytest to ignore legacy test dir
  • Added --cov=cal_tools and --cov=xfel_calibrate to get coverage output for both modules
  • Added ssh key variable to CI, added .setup-ssh-euxfel-gitlab template to CI
• Change internal dependencies to use SSH instead of files (this is now possible as xsoft has a keypair on gitlab)

These changes make the installation of the package a bit more standard, removes the step of having to run two install commands (one for requirements.txt, and one for .), and let us use extras to define additional optional dependencies (e.g. test, dev, etc...).

How Has This Been Tested?

Tested locally by running pip install without extras, with test, with dev, and with test,dev, then ran the pytests to check that everything was still working correctly.

Also updated the CI definition to use the new setup style, so the tests also executed with the pipeline.

Relevant Documents (optional)

Types of changes

• Bug fix (non-breaking change which fixes an issue)

Checklist:

• My code follows the code style of this project.
• My change requires a change to the documentation.
• I have updated the documentation accordingly.

Reviewers

@danilevc @ahmedk @hammerd

Notes:

For our local dependencies the name of the package doesn't match the name of the repository, so I have to explicitly write out the name, and then use the git ssh url as a target, e.g. iCalibrationDB @ git+ssh://git@git.xfel.eu:10022/detectors/cal_db_interactive.git@2.0.1. @kluyvert does this look correct?

.gitlab-ci.yml

@@ -6,16 +6,20 @@ checks:
   stage: check
   only: [merge_requests]
   allow_failure: true
+  before_script:
+    - python3 -m venv .venv
+    - source .venv/bin/activate
+    - python3 -m pip install --upgrade pip
   script:
     - export PATH=/home/gitlab-runner/.local/bin:$PATH
-    # We'd like to run the pre-commit hooks only on files that are being
-    # modified by this merge request, however
-    # `CI_MERGE_REQUEST_TARGET_BRANCH_SHA` is a 'premium' feature according to
-    # GitLab... so this is a workaround for extracting the hash
+    #  We'd like to run the pre-commit hooks only on files that are being
+    #  modified by this merge request, however
+    #  `CI_MERGE_REQUEST_TARGET_BRANCH_SHA` is a 'premium' feature according to
+    #  GitLab... so this is a workaround for extracting the hash
     - export CI_MERGE_REQUEST_TARGET_BRANCH_SHA=$(git ls-remote origin $CI_MERGE_REQUEST_TARGET_BRANCH_NAME | cut -d$'\t' -f1)
-    - export FILES=$(git diff $CI_COMMIT_SHA  $CI_MERGE_REQUEST_TARGET_BRANCH_SHA --name-only | tr '\n' ' ')
+    - export FILES=$(git diff $CI_COMMIT_SHA $CI_MERGE_REQUEST_TARGET_BRANCH_SHA --name-only | tr '\n' ' ')
     - python3 -m pip install ".[test,dev]"
-    - echo "Running pre-commit on diff from  $CI_COMMIT_SHA to $CI_MERGE_REQUEST_TARGET_BRANCH_SHA ($CI_MERGE_REQUEST_TARGET_BRANCH_NAME)"
+    - echo "Running pre-commit on diff from $CI_COMMIT_SHA to $CI_MERGE_REQUEST_TARGET_BRANCH_SHA ($CI_MERGE_REQUEST_TARGET_BRANCH_NAME)"
     #  Pass list of modified files to pre-commit so that it only checks them
     - echo $FILES | xargs pre-commit run --color=always --files
 
@@ -34,9 +38,9 @@ pytest:
     - ssh-keyscan -p 10022 git.xfel.eu > $CI_PROJECT_DIR/.ssh/known_hosts
     - ls  $CI_PROJECT_DIR/.ssh
     - echo $GIT_SSH_COMMAND
-  script:
     - python3 -m venv .venv
     - source .venv/bin/activate
     - python3 -m pip install --upgrade pip
+  script:
     - python3 -m pip install ".[test]"
     - python3 -m pytest --cov=cal_tools --cov=xfel_calibrate  --ignore=tests/legacy