Make constant tables in both markdown & latex format

Description

This is an alternative to !1034 (closed). Converting markdown tables to rst (to latex to pdf) is going wrong, so we want a workaround.

This uses Jupyter's machinery to produce both formats simultaneously, and store both in the notebook file. In the live notebook, markdown has higher priority (being based on HTML), so the Latex output is hidden. When converting to rst for the PDF report, the Latex format is preferred.

I've drafted this with the calcat_interface2 module, but we could extend it to anywhere we're generating tables.

How Has This Been Tested?

For now, just in a small test notebook. End to end tests still to be done.

Relevant Documents (optional)

Output snippet from nbconvert --to rst

.. code:: ipython3

    agipd_cd.summary_table()




.. math::

    \begin{tabular}{rll}
    \hline
       Modules & Offset                                                                                       & SlopesFF                                                                                     \\
    \hline
             0 & \href{https://in.xfel.eu/calibration/calibration_constant_versions/141880}{2022-09-02 07:42} & \href{https://in.xfel.eu/calibration/calibration_constant_versions/118111}{2022-02-05 13:07} \\
             1 & \href{https://in.xfel.eu/calibration/calibration_constant_versions/141821}{2022-09-02 07:42} & \href{https://in.xfel.eu/calibration/calibration_constant_versions/118138}{2022-02-05 13:07} \\

Types of changes

• New feature (non-breaking change which adds functionality)

Checklist:

• My code follows the code style of this project.

Reviewers

@ahmedk @schmidtp

mentioned in merge request !1034 (closed)

Nice, the xarray implementation that you mentioned previously.

I guess you plan to update calcat_interface as well with the same changes after testing. As this is the currently used API in production.

Anyhow I realize that this is still a Draft and I thank you for this neat solution.

added 1 commit

• 0c3108a8 - Use DisplayTables in calcat_interface

Compare with previous version

Tested with an AGIPD example:

• Notebook on max-jhub
  • When you first open it, you'll see the latex tables. Click the 'Not trusted' notification to mark the notebook as trusted, then reload, and the markdown tables appear. That's because Markdown can contain arbitrary HTML, HTML can contain Javascript, and Javascript on a notebook page can talk to a kernel.
• PDF at /gpfs/exfel/data/scratch/kluyvert/agipd-corr-p900466-r128/MID_DET_AGIPD1M-1-CORRECT-240730_100527.041493.pdf

In the notebook (once trusted):

In the PDF:

Versus the PDF from the same run corrected by the deployed system:

I checked as well the finished tests and reports(and notebooks) look neat.

added 1 commit

• 05f45f1e - Try to tex-escape text inside tables

Compare with previous version

Unrelated to this MR, I think it might be useful to use enums for this as its usage is expanding.

For APIs we're exposing to scientists, I prefer to do options like this as strings rather than an enum. I think enums are good for experienced software developers, but they're an extra complication to think about if you're not used to them, and Python has a lot of cases already where we use strings to pick from a finite set of choices.

We could pass a string in and use an enum to validate it internally. But I'm not sure that's worth it.

Make sense, so my focus was more on avoiding string duplication in code. I guess a simple way to avoid this is to define Global constant for them that we check against. And we can leave of course at least the default string values in the method definitions if this will have a better usability.

TIL, clever! LGTM

added 1 commit

• 61da3234 - Sync DisplayTables implementation back to calcat_interface2

Compare with previous version

LGTM

merged

mentioned in commit 80f0e8a2

Thanks for the review

changed milestone to %3.15.0

mentioned in merge request !968 (merged)