[pnCCD] [CORRECT][DARK] EXtra-data and pasha
- Add Extra data for reading raw data for CORRECT and DARK notebooks.
- Using pasha in parallelizing data processing instead of pyDetLib.
- Removing all pyDetlib function, instead for common-mode and plotting.
- Use balance_sequences for pnccd correction.
- Performance wise. The PnCCD gained a lot from these changes. The correct notebook was not doing any parallelization over images or sequence files.
It went from 3 hours for correcting 12 sequences at /gpfs/exfel/exp/SQS/202031/p900166/raw/r0347
to about 7 mins, after using parallelization over sequence files and pasha over images.
Description
How Has This Been Tested?
Correct
Production case comparison (unfair one)
Fair comparison of correcting one sequence only.
MASTER one sequences correction
Extra-data one sequences correction
DARK [x5 speed up]
NOTE: data quality wise. Noise generated constants are the same after specifying relative tolerance parameter to 1e-4
Relevant Documents (optional)
Types of changes
- New feature (non-breaking change which adds functionality)
- Refactor (refactoring code with no functionality changes)
Checklist:
Reviewers
Edited by Karim Ahmed