diff --git a/README.rst b/README.rst index 2f14a98a24a722d37d5c6aa967318f63cfc453f2..8064819d434ecde8a0f393653dd1d231f1a93b3d 100644 --- a/README.rst +++ b/README.rst @@ -6,6 +6,8 @@ The offline calibration is a package that consists of different services, responsible for applying most of the offline calibration and characterization for the detectors. +.. contents:: + Offline Calibration Installation ******************************** @@ -71,7 +73,7 @@ venv) activate the virtual environment first, and then run: .. code:: python3 -m pip install ipykernel # If not using a venv add `--user` flag - python3 -m ipykernel install --user --name pcalibration --display-name "pcalibration" # If not using a venv pick different name + python3 -m ipykernel install --user --name pycalibration --display-name "pycalibration" # If not using a venv pick different name This can be useful for Jupyter notebook tools as https://max-jhub.desy.de/hub/login @@ -101,16 +103,71 @@ To do this check the settings page here: https://git.xfel.eu/gitlab/profile/keys Pre-Commit Hooks ================ -This repository uses pre-commit hooks to ensure that a number of checks have -been made before code is committed and pushed to the... +This repository uses pre-commit hooks automatically run some code quality and +standard checks, this includes the following: + +a. ``identity`` - The 'identity' meta hook prints off a list of files that the hooks will execute on +b. 'Standard' file checks + + 1. ``check-added-large-files`` - Ensures no large files are committed to repo + 2. ``check-ast`` - Checks that the python AST is parseable + 3. ``check-json`` - Checks json file formatting is parseable + 4. ``check-yaml`` - Checks yaml file formatting is parseable + 5. ``check-toml`` - Checks toml file formatting is parseable + 6. ``rstcheck`` - Checks rst file formatting is parseable + 7. ``end-of-file-fixer`` - Fixes EoF to be consistent + 8. ``trailing-whitespace`` - Removes trailing whitespaces from lines + 9. ``check-merge-conflict`` - Checks no merge conflicts remain in the commit + 10. ``mixed-line-ending`` - Fixes mixed line endings + +c. Code checks + + 1. ``flake8`` - Code style checks + 2. ``isort`` - Sorts imports in python files + 3. ``check-docstring-first`` - Ensures docstrings are in the correct place + +d. Notebook checks + + 1. ``nbqa-flake8`` - Runs flake8 on notebook cells + 2. ``nbqa-isort`` - Runs isort on notebook cells + 3. ``nbstripoutput`` - Strips output from ipynb files + +To install these checks, set up you environment as mentioned above and then run +the command: + +.. code:: + + pre-commit install-hooks + +This will set up the hooks in git locally, so that each time you run the command +``git commit`` the hooks get executed on the **staged files only**, beware that +if the pre-commit hooks find required changes some of them will **modify your +files**, however they only modify the current working files, not the ones you +have already staged. This means that you can look at the diff between your +staged files and the ones that were modified to see what changes are suggested. + + +Run Checks Only On Diffs +------------------------ + +Typically ``pre-commit`` is ran on ``--all-files`` within a CI, however as this +is being set up on an existing codebase these checks will always fail with a +substantial number of issues. Using some creative workarounds, the CI has been +set up to only run on files which have changed between a PR and the target +branch. -TODO: +If you want to run the pre-commit checks as they would run on the CI, then you +can use the ``bin/pre-commit-diff.sh`` to execute the checks as on the CI +pipeline. -1. List of checks -2. Mention how the CI only runs on diffs between PR branch and target branch -3. Include runnable script to have same behaviour as on gitlab -4. Mention having to rebase for diffs to be correct if merge target changes +A side effect of this is that the checks will run on **all** of the differences +between the 'local' and target branch. This means that if changes have recently +been merged into the target branch, and there is divergence between the two, +then the tests will run on all the differences. +If this happens and the hooks in the CI (or via the script) run on the wrong +files then you should **rebase onto the target branch** to prevent the checks +from running on the wrong files/diffs. Python Scripted Calibration