SCS Toolbox
===========

Installation
------------

Recommended: Proposal Environment setup
+++++++++++++++++++++++++++++++++++++++

The proposal specific environment is installed by launching in a shell
terminal on Maxwell:

.. code:: bash

    module load exfel exfel-python
    scs-activate-toolbox-kernel --proposal 2780

where in this example 2780 is the proposal number. After this and refreshing the
browser, a new kernel named ``SCS Toolbox (p002780)`` is available and should
be used to run jupyter notebooks on the Maxwell Jupyter hub.

Figures setup: enabling matplotlib constrained layout
+++++++++++++++++++++++++++++++++++++++++++++++++++++

To get the best looking figures generated by the SCS Toolbox, you
need to enable the experimental constrained_layout_ solver in matplotlib. This
is done in jupyter notebook with adding at the start the following lines:

.. code:: python

   import matplotlib.pyplot as plt
   plt.rcParams['figure.constrained_layout.use'] = True

.. _constrained_layout: https://matplotlib.org/stable/tutorials/intermediate/constrainedlayout_guide.html


Alternative: Manual ToolBox Installation
++++++++++++++++++++++++++++++++++++++++

The ToolBox may be installed in any environment. However, it depends on the
extra_data and the euxfel_bunch_pattern package, which are no official third
party python modules. Within environments where the latter are not present, they
need to be installed by hand.

In most cases, you will want to install it in the *exfel_python* environment
which is the one corresponding to the *xfel* kernel on max-jhub.

To do so, first activate that environment:

.. code:: bash

    module load exfel exfel-python
    
Then, check that the scs_toolbox is not already installed:

.. code:: bash

    pip show toolbox_scs

If the toolbox has been installed in your home directory previously, everything
is set up. If you need to upgrade the toolbox to a more recent version, you
have to either uninstall the current version:

.. code:: bash

    pip uninstall toolbox_scs

or if it was installed in development mode, go in the toolbox directory and
pull the last commits from git:

.. code:: bash

    cd #toolbox top folder
    git pull

Otherwise it needs to be installed (only once). In that you first need to
clone the toolbox somewhere (in your home directory for example) and install
the package. Here the -e flag install the package in development mode, which
means no files are copied but only linked such that any changes made to the
toolbox files will have effect:

.. code:: bash

    cd ~
    git clone https://git.xfel.eu/SCS/ToolBox.git
    cd ~/ToolBox
    pip install --user -e ".[maxwell]"


Transferring Data
-----------------

The DAQ system save data on the online cluster. To analyze data on the
Maxwell offline cluster, they need to be transferred there. This is achieved by
login at:

https://in.xfel.eu/metadata

and then navigating to the proposal, then to the ``Runs`` tab from where runs can
be transferred to the offline cluster by marking them as ``Good`` in the
``Data Assessment``. Depending on the amount of data in the run, this can take a
while.

.. image:: metadata.png

Processing Data
---------------

On the Maxwell Jupyter hub:

https://max-jhub.desy.de

notebooks can be executed using the corresponding Proposal Environment
kernel or the ``xfel`` kernel if the toolbox was manually installed. For quick
startup, example notebooks (.ipynb files) can be directly downloaded from the
:ref:`How to's` section by clicking on the ``View page source``.

.. toctree::
   :caption: Contents:
   :maxdepth: 2
   :hidden:

   howtos.rst
   maintainers.rst
   changelog.rst


Contribute
~~~~~~~~~~

For reasons of readability, contributions preferrably comply with the PEP8_ code structure guidelines.

.. _PEP8: https://www.python.org/dev/peps/pep-0008/#a-foolish-consistency-is-the-hobgoblin-of-little-minds 

The associated code checker, called 'flake8', can be installed via PyPi.