Feat/pre commit checks

• Added pre-commit hooks which:
  • isort on .py and .ipynb
  • flake8 on .py and .ipynb
  • nbstripout
  • rstcheck
  • check-added-large-files
  • check-ast
  • check-json
  • check-yaml
  • check-toml
  • end-of-file-fixer
  • trailing-whitespace
  • check-docstring-first
  • check-merge-conflict
  • mixed-line-ending
• Hooks run as part of the check stage in the CI
• Fixed some of the issues:
  • EoF
  • Trailing whitespace
  • isort (but only for .py)

I included a lot of other changes and fixes in the original MR (https://git.xfel.eu/gitlab/detectors/pycalibration/merge_requests/411) before figuring out how to get the checks to run only on changed files, which meant that the MR implementing pre-commit checks also refactored a decent chunk of code. It would be better to do that refactoring in separate MRs so I've closed the original one and opened this to replace it.

@ahmedk @danilevc @hammerd feel free to review when you have time

TODO:

• Add pre-commit to requirements.txt
• Don't mutate the code during pre-commit checks - changed my mind about this a few times, left it to mutate the code as that is only done on staged files, so any mutations can be reverted by undoing changes anyway. Added an explanation on this to the readme instead.
• Add info to the readme and to the wiki
• Add explanation of what each of the checks is for in more detail
• Add scripts to execute w/ diff-only (same as the CI pipeline)

.gitlab-ci.yml

@@ -7,10 +7,15 @@ checks:
   only: [merge_requests]
   script:
     - export PATH=/home/gitlab-runner/.local/bin:$PATH
+    # We'd like to run the pre-commit hooks only on files that are being
+    # modified by this merge request, however
+    # `CI_MERGE_REQUEST_TARGET_BRANCH_SHA` is a 'premium' feature according to
+    # GitLab... so this is a workaround for extracting the hash
     - export CI_MERGE_REQUEST_TARGET_BRANCH_SHA=$(git ls-remote origin $CI_MERGE_REQUEST_TARGET_BRANCH_NAME | cut -d$'\t' -f1)
     - export FILES=$(git diff $CI_COMMIT_SHA  $CI_MERGE_REQUEST_TARGET_BRANCH_SHA --name-only | tr '\n' ' ')
     - echo "Running pre-commit on diff from  $CI_COMMIT_SHA to $CI_MERGE_REQUEST_TARGET_BRANCH_SHA ($CI_MERGE_REQUEST_TARGET_BRANCH_NAME)"
     - python3 -m pip install --user -r requirements.txt
+    #  Pass list of modified files to pre-commit so that it only checks them
     - echo $FILES | xargs pre-commit run --color=always --files
 
 pytest: