Skip to content
Snippets Groups Projects
Commit 3cd587f1 authored by Loïc Le Guyader's avatar Loïc Le Guyader
Browse files

Merge branch 'install' into 'master'

Improve installation instruction

Closes #30

See merge request !145
parents d32d4d6b 3cf5277f
No related branches found
No related tags found
1 merge request!145Improve installation instruction
###########
SCS ToolBox
###########
Kernel
######
The SCS ToolBox is design to work in the exfel_anaconda3 environement. This can
be selected on the online cluster by:
`module load exfel exfel_anaconda3`
before launching the jupyter-notebook or on max-jhub by selecting the 'xfel'
kernel instead of the 'Python 3' anaconda environement maintained by DESY.
Installation
############
As long as the ToolBox is not yet added to the exfel_anaconda3 environment it needs to be installed locally.
Activate environment mentioned above and check installation of scs_toolbox:
.. code:: bash
pip show toolbox_scs
If the toolbox has been installed in your home directory previously, everything is set up. Otherwise it needs to be installed (only once). In that case enter the following command in the directory where the *setup.py* script is located:
.. code:: bash
pip install --user .
If you intend to develop code in the toolbox use the -e flag for installation. This creates a symbolic link to the source code you are working on.
.. code:: bash
pip install --user -e .
Documentation
#############
......
Getting started
~~~~~~~~~~~~~~~
Toolbox Installation
--------------------
Recommended: Proposal Environment setup
+++++++++++++++++++++++++++++++++++++++
The proposal specific environment is installed by launching in a shell
terminal on Maxwell:
.. code:: bash
module load exfel exfel_anaconda3
scs-activate-toolbox-kernel --proposal 2780
where in this example 2780 is the proposal number. After this and refreshing the
browser, a new kernel named ``SCS Toolbox (p002780)`` is available and should
be used to run jupyter notebooks on the Maxwell Jupyter hub.
Alternative: Manual ToolBox Installation
++++++++++++++++++++++++++++++++++++++++
The ToolBox may be installed in any environment. However, it depends on the
extra_data and the euxfel_bunch_pattern package, which are no official third
party python modules. Within environments where the latter are not present, they
need to be installed by hand.
Furthermore, as long as the ToolBox is not yet added to one of our custom
environments, it needs to be installed locally. Activate your preferred
environment and check installation of scs_toolbox by typing:
.. code:: bash
pip show toolbox_scs
If the toolbox has been installed in your home directory previously, everything
is set up. Otherwise it needs to be installed (only once). In that case enter
following command from the the ToolBox top-level directory:
.. code:: bash
pip install --user ".[maxwell]"
Alternatively, use the -e flag for installation to install the package in
development mode.
.. code:: bash
pip install --user -e ".[maxwell]"
Transferring Data
-----------------
The DAQ system save data on the online cluster. To analyze data on the
Maxwell offline cluster, they need to be transferred there. This is achieved by
login at:
https://in.xfel.eu/metadata
and then navigating to the proposal, then to the ``Runs`` tab from where runs can
be transferred to the offline cluster by marking them as ``Good`` in the
``Data Assessment``. Depending on the amount of data in the run, this can take a
while.
.. image:: metadata.png
Processing Data
---------------
On the Maxwell Jupyter hub:
https://max-jhub.desy.de
notebooks can be executed using the corresponding Proposal Environment
kernel or the ``xfel`` kernel if the toolbox was manually installed. For quick
startup, example notebooks (.ipynb files) can be directly downloaded from the
:ref:`How to's` section by clicking on the ``View page source``.
Welcome to SCS Toolbox's documentation!
=======================================
SCS Tollbox
===========
.. toctree::
:maxdepth: 2
:caption: Contents:
:numbered:
:titlesonly:
:glob:
:hidden:
Installation
------------
getting_started.rst
howtos.rst
Recommended: Proposal Environment setup
+++++++++++++++++++++++++++++++++++++++
The proposal specific environment is installed by launching in a shell
terminal on Maxwell:
``Contribute``
~~~~~~~~~~~~~~
.. code:: bash
For reasons of readability, contributions preferrably comply with the PEP8_ code structure guidelines.
module load exfel exfel_anaconda3
scs-activate-toolbox-kernel --proposal 2780
.. _PEP8: https://www.python.org/dev/peps/pep-0008/#a-foolish-consistency-is-the-hobgoblin-of-little-minds
where in this example 2780 is the proposal number. After this and refreshing the
browser, a new kernel named ``SCS Toolbox (p002780)`` is available and should
be used to run jupyter notebooks on the Maxwell Jupyter hub.
The associated code checker, called 'flake8', can be installed via PyPi.
Alternative: Manual ToolBox Installation
++++++++++++++++++++++++++++++++++++++++
The ToolBox may be installed in any environment. However, it depends on the
extra_data and the euxfel_bunch_pattern package, which are no official third
party python modules. Within environments where the latter are not present, they
need to be installed by hand.
In most cases, you will want to install it in the *exfel_anaconda3* environment
which is the one corresponding to the *xfel* kernel on max-jhub.
To do so, first activate that environment:
.. code:: bash
module load exfel exfel_anaconda3
Then, check that the scs_toolbox is not already installed:
.. code:: bash
pip show toolbox_scs
If the toolbox has been installed in your home directory previously, everything
is set up. If you need to upgrade the toolbox to a more recent version, you
have to either uninstall the current version:
.. code:: bash
pip uninstall toolbox_scs
or if it was installed in development mode, go in the toolbox directory and
pull the last commits from git:
.. code:: bash
cd #toolbox top folder
git pull
Otherwise it needs to be installed (only once). In that you first need to
clone the toolbox somewhere (in your home directory for example) and install
the package. Here the -e flag install the package in development mode, which
means no files are copied but only linked such that any changes made to the
toolbox files will have effect:
Module index
============
.. code:: bash
*to to* (automatized doc generation)
cd ~
git clone https://git.xfel.eu/SCS/ToolBox.git
cd ~/ToolBox
pip install --user -e ".[maxwell]"
**toolbox_scs**: Top-level entry point
**detectors**: detector specific routines
Transferring Data
-----------------
**routines**: Automatized evaluations involving several instruments
The DAQ system save data on the online cluster. To analyze data on the
Maxwell offline cluster, they need to be transferred there. This is achieved by
login at:
**misc**: Various sub-routines and helper functions
https://in.xfel.eu/metadata
**test**: Test environment
and then navigating to the proposal, then to the ``Runs`` tab from where runs can
be transferred to the offline cluster by marking them as ``Good`` in the
``Data Assessment``. Depending on the amount of data in the run, this can take a
while.
**util**: Package related routines
.. image:: metadata.png
Processing Data
---------------
On the Maxwell Jupyter hub:
https://max-jhub.desy.de
notebooks can be executed using the corresponding Proposal Environment
kernel or the ``xfel`` kernel if the toolbox was manually installed. For quick
startup, example notebooks (.ipynb files) can be directly downloaded from the
:ref:`How to's` section by clicking on the ``View page source``.
Documentation contents
----------------------
.. toctree::
:caption: Contents:
:maxdepth: 2
howtos.rst
Contribute
~~~~~~~~~~
For reasons of readability, contributions preferrably comply with the PEP8_ code structure guidelines.
.. _PEP8: https://www.python.org/dev/peps/pep-0008/#a-foolish-consistency-is-the-hobgoblin-of-little-minds
The associated code checker, called 'flake8', can be installed via PyPi.
Indices and tables
==================
......
0% Loading or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment