SCS Toolbox
===========

Installation
------------

Recommended: Proposal Environment setup
+++++++++++++++++++++++++++++++++++++++

The proposal specific environment is installed by launching in a shell
terminal on Maxwell:

.. code:: bash

module load exfel exfel_anaconda3
scs-activate-toolbox-kernel --proposal 2780

where in this example 2780 is the proposal number. After this and refreshing the
browser, a new kernel named ``SCS Toolbox (p002780)`` is available and should
be used to run jupyter notebooks on the Maxwell Jupyter hub.

Alternative: Manual ToolBox Installation
++++++++++++++++++++++++++++++++++++++++

The ToolBox may be installed in any environment. However, it depends on the
extra_data and the euxfel_bunch_pattern package, which are no official third
party python modules. Within environments where the latter are not present, they
need to be installed by hand.

In most cases, you will want to install it in the *exfel_anaconda3* environment
which is the one corresponding to the *xfel* kernel on max-jhub.

To do so, first activate that environment:

.. code:: bash

module load exfel exfel_anaconda3

Then, check that the scs_toolbox is not already installed:

.. code:: bash

pip show toolbox_scs

If the toolbox has been installed in your home directory previously, everything
is set up. If you need to upgrade the toolbox to a more recent version, you
have to either uninstall the current version:

.. code:: bash

pip uninstall toolbox_scs

or if it was installed in development mode, go in the toolbox directory and
pull the last commits from git:

.. code:: bash

cd #toolbox top folder
git pull

Otherwise it needs to be installed (only once). In that you first need to
clone the toolbox somewhere (in your home directory for example) and install
the package. Here the -e flag install the package in development mode, which
means no files are copied but only linked such that any changes made to the
toolbox files will have effect:

.. code:: bash

cd ~
git clone https://git.xfel.eu/SCS/ToolBox.git
cd ~/ToolBox
pip install --user -e ".[maxwell]"

Transferring Data
-----------------

The DAQ system save data on the online cluster. To analyze data on the
Maxwell offline cluster, they need to be transferred there. This is achieved by
login at:

https://in.xfel.eu/metadata

and then navigating to the proposal, then to the ``Runs`` tab from where runs can
be transferred to the offline cluster by marking them as ``Good`` in the
``Data Assessment``. Depending on the amount of data in the run, this can take a
while.

.. image:: metadata.png

Processing Data
---------------

On the Maxwell Jupyter hub:

https://max-jhub.desy.de

notebooks can be executed using the corresponding Proposal Environment
kernel or the ``xfel`` kernel if the toolbox was manually installed. For quick
startup, example notebooks (.ipynb files) can be directly downloaded from the
:ref:`How to's` section by clicking on the ``View page source``.

Documentation contents
----------------------

.. toctree::
:caption: Contents:
:maxdepth: 2

howtos.rst
maintainers.rst
changelog.rst

Contribute
~~~~~~~~~~

For reasons of readability, contributions preferrably comply with the PEP8_ code structure guidelines.

.. _PEP8: https://www.python.org/dev/peps/pep-0008/#a-foolish-consistency-is-the-hobgoblin-of-little-minds

The associated code checker, called 'flake8', can be installed via PyPi.

Indices and tables
==================

* :ref:`genindex`
* :ref:`modindex`
* :ref:`search`