Review by Danilo.
Implement comments from Danilo:
- https://git.xfel.eu/dataAnalysis/spi-data-reduction/-/blob/main/src/spi_reduction/utilities.py#L19 -- it would be nice to raise an exception if the path is not in the GPFS structure that is expected. The directory called p1234 could be anywhere, or the regexp match can fail. In that case, the error message is not so understandable.
- https://git.xfel.eu/dataAnalysis/spi-data-reduction/-/blob/main/src/spi_reduction/exdf_entry.py really has less documentation and it should be the entry point if I understood it correctly.
- In the same token as before, I don't see much error-handling here: https://git.xfel.eu/dataAnalysis/spi-data-reduction/-/blob/main/src/spi_reduction/user_frames_select.py#L355
- https://git.xfel.eu/dataAnalysis/spi-data-reduction/-/blob/main/src/spi_reduction/user_frames_select.py#L137 Sometimes the number of CPU counts is not really reliable. OFten I let it decide automatically, which works better in general. Also, using ThreadPoolExecutor has worked more reliably for me in the past (see https://docs.python.org/3/library/multiprocessing.html#multiprocessing.pool.ThreadPool and https://docs.python.org/3/library/concurrent.futures.html#concurrent.futures.ThreadPoolExecutor and https://stackoverflow.com/questions/73351280/what-is-the-difference-between-threadpoolexecutor-and-threadpool). Does not really matter, but it could make the multithreading more reliable in Virtual Machines (like VISA), where the number of CPU counts are less reliable.