From 5513aa10ffc17a0034ecbd0791880cc15b83d9fb Mon Sep 17 00:00:00 2001
From: Steffen Hauf <haufs@max-exfl015.desy.de>
Date: Fri, 22 Jun 2018 15:52:41 +0200
Subject: [PATCH] Update docs
---
bin/slurm_calibrate.sh | 2 +-
docs/source/advanced.rst | 2 +-
docs/source/conf.py | 2 +-
docs/source/configuration.rst | 2 +-
docs/source/how_it_works.rst | 7 +++-
docs/source/installation.rst | 39 +++++++++++++++++--
docs/source/workflow.rst | 18 ++++++---
...PD_FlatField_Radial_per_pixel_CI_NBC.ipynb | 4 +-
8 files changed, 59 insertions(+), 17 deletions(-)
diff --git a/bin/slurm_calibrate.sh b/bin/slurm_calibrate.sh
index 69d1a1ecb..9b757db11 100755
--- a/bin/slurm_calibrate.sh
+++ b/bin/slurm_calibrate.sh
@@ -63,4 +63,4 @@ if [ "${final}" == "FINAL" ]
then
${finalize}
fi
-killall -9 cal_influx_feeder.sh
+killall -9 cal_influx_feeder.sh || true
diff --git a/docs/source/advanced.rst b/docs/source/advanced.rst
index 7b760fb2c..c9c707b1f 100644
--- a/docs/source/advanced.rst
+++ b/docs/source/advanced.rst
@@ -107,5 +107,5 @@ This can be useful to add user requests while running. For this:
5. Any runs which did not correct do to failures of the live edit can then be relaunched
manually, assuming the correction notebook allows run and overwrite paramters::
- python calibrate.py ...... --run XYZ,ZXY-YYS --overwrite
+ xfel-calibrate ...... --run XYZ,ZXY-YYS --overwrite
diff --git a/docs/source/conf.py b/docs/source/conf.py
index 5514a507c..2d12d2d83 100644
--- a/docs/source/conf.py
+++ b/docs/source/conf.py
@@ -349,7 +349,7 @@ intersphinx_mapping = {'https://docs.python.org/': None}
-# first install pandoc if necessary
+# first install pandoc if necessary, this is mainly meant for the RTD builds
try:
from nbconvert.utils.pandoc import check_pandoc_version
check_pandoc_version()
diff --git a/docs/source/configuration.rst b/docs/source/configuration.rst
index 8d1e5fc4d..26d0d8a3c 100644
--- a/docs/source/configuration.rst
+++ b/docs/source/configuration.rst
@@ -47,7 +47,7 @@ A comment is given for the meaning of each configuration parameter.
Notebooks
---------
-The `calibrate.py` script will expose any notebooks that are configured here to the
+The `xfel-calibrate` tool will expose any notebooks that are configured here to the
command line by automatically parsing the parameters given in the notebooks first cell.
The configuration is to be given in form of a python directory::
diff --git a/docs/source/how_it_works.rst b/docs/source/how_it_works.rst
index 82ba28d46..d8246c188 100644
--- a/docs/source/how_it_works.rst
+++ b/docs/source/how_it_works.rst
@@ -4,7 +4,12 @@ How it Works
============
The European XFEL Offline Calibration utilizes the tools nbconvert_ and nbparameterise_
-to expose Jupyter_ notebooks to a command line interface.
+to expose Jupyter_ notebooks to a command line interface. In the process reports are generated
+from these notebooks. The general interface is::
+
+ % xfel-calibrate DETECTOR TYPE
+
+where `DETECTOR` and `TYPE` specify the task to be performed.
Additionally, it leverages the DESY/XFEL Maxwell cluster to run these jobs in parallel
via SLURM_.
diff --git a/docs/source/installation.rst b/docs/source/installation.rst
index b329b891f..406b7e1d0 100644
--- a/docs/source/installation.rst
+++ b/docs/source/installation.rst
@@ -12,14 +12,16 @@ Prerequisites
Prerequisites can be categorized into three types:
1. those required to run the the general tool chain,
-2. and those required by the notebooks themselves,
-3. those required to run notebooks concurrently on a HPC cluster.
+2. those required by the notebooks themselves,
+3. and those required to run notebooks concurrently on a HPC cluster.
-Categories 1+2 are Python modules, as given in the `requirements.txt`
+Categories 1+2 are usually Python modules, as given in the `requirements.txt`
file found in the root directory. Additionally, PyDetLib_ is required.
It can be installed either as a Karabo dependency or manually.
Parallel execution is currently supported in a `SLURM_` environment.
+However, sequential execution is available using the `--no-cluster-job`
+parameter when executing `xfel-calibrate`.
.. _Python: http://www.python.org/
.. _Karabo: https://in.xfel.eu/readthedocs/docs/karabo/en/latest/
@@ -40,4 +42,33 @@ all prerequisites and this tool chain by running::
pip install -r requirements.txt .
After installation, you should adjust the :ref:`configuration` and settings to
-match your environment.
\ No newline at end of file
+match your environment.
+
+The toolchain is then available via the::
+
+ xfel-calibrate
+
+command.
+
+Development Installation
+------------------------
+
+For a development installation, which automatically picks up (most) changes,
+first install the dependencies as above, but then install the took-chain
+separately in development mode::
+
+ pip install -e .
+
+
+
+Installation of New Notebooks
+-----------------------------
+
+To install new, previously untracked notebooks repeat the installation of
+the the tool-chain, without prerequisites, from the package base directory::
+
+ pip install --upgrade .
+
+Or, in case you are activily developing::
+
+ pip install -e .
\ No newline at end of file
diff --git a/docs/source/workflow.rst b/docs/source/workflow.rst
index f52794d6d..ab3bc6172 100644
--- a/docs/source/workflow.rst
+++ b/docs/source/workflow.rst
@@ -104,8 +104,8 @@ for parallelisation, define your profile name in this variable.
The excerpt above is from a flat field characterization notebook for AGIPD. The code would lead
to the following parameters being exposed via the command line::
- % python calibrate.py AGIPD FF --help
- usage: calibrate_nbc.py [-h] --in-folder str [--modules str [str ...]]
+ % xfel-calibrate AGIPD FF --help
+ usage: xfel-calibrate.py [-h] --in-folder str [--modules str [str ...]]
--out-folder str --runs str [str ...]
[--sequences str [str ...]] [--cluster-profile str]
[--local-output] [--db-output] [--bias-voltage int]
@@ -300,19 +300,25 @@ to the list of notebooks as mentioned in the :ref:`configuration` section.
Any changes you now make in the notebook will be automatically propagated to the command line.
Specifically, you should verify that all arguments are parsed correctly, e.g. by calling::
- python calibrate.py DETECTOR NOTEBOOK_TYPE --help
+ xfel-calibrate DETECTOR NOTEBOOK_TYPE --help
From then on, check include if parallel slurm jobs are exectuted correctly and if a report
is generated at the end.
Finally, you should verify that the report contains the information you'd like to convey and
-is intelegable to people other than you.
+is inteligable to people other than you.
+
+.. note::
+
+ You can run the `xfel-calibrate` command without starting a SLURM cluster job, giving
+ you direct access to console output, by adding the `--no-cluster-job` option.
Documenting
-----------
-Most documentation should be done in the notebook itself. However, please add new notebooks
-to the `available_notebooks.rst` file in the `docs` folder.
+Most documentation should be done in the notebook itself. Any notebooks specified in the
+`notebook.py` file will automatically show up in the :ref:`available_notebooks` section of this
+documentation.
.. _nbparameterise: https://github.com/takluyver/nbparameterise
diff --git a/notebooks/LPD/LPD_FlatField_Radial_per_pixel_CI_NBC.ipynb b/notebooks/LPD/LPD_FlatField_Radial_per_pixel_CI_NBC.ipynb
index 56dff9bac..a04caeffe 100644
--- a/notebooks/LPD/LPD_FlatField_Radial_per_pixel_CI_NBC.ipynb
+++ b/notebooks/LPD/LPD_FlatField_Radial_per_pixel_CI_NBC.ipynb
@@ -4,7 +4,7 @@
"cell_type": "markdown",
"metadata": {},
"source": [
- "# LPD Radial \"Flat Field\" Evaluation #\n",
+ "# LPD Radial X-ray Gain Evaluation #\n",
"\n",
"Author: S. Hauf, Version 0.5\n",
"\n",
@@ -15,7 +15,7 @@
},
{
"cell_type": "code",
- "execution_count": 215,
+ "execution_count": 1,
"metadata": {
"collapsed": true
},
--
GitLab