[DataFile] Add parallel dataset compression (!753) · Merge requests · calibration / pycalibration · GitLab

Snippets Groups Projects

Merged Philipp Schmidt requested to merge feat/datafile-parallel-compress into master 2 years ago

All threads resolved!

Description

For a while now, we are using the parallelized compression offered by write_compressed_frames. This MR adds this capability to the DataFrame API to write such compressed datasets side-by-side with uncompressed ones.

How Has This Been Tested?

Running LPD_Correct_Fast with default arguments.

Types of changes

Refactor (refactoring code with no functionality changes)

Reviewers

Activity

Philipp Schmidt added Waiting for review label 2 years ago

added Waiting for review label
Karim Ahmed @ahmedk started a thread on an old version of the diff 2 years ago

Resolved 2 years ago by Philipp Schmidt
Last reply by Philipp Schmidt 2 years ago

Philipp Schmidt added 1 commit 2 years ago

added 1 commit

e9b9aacf - Add parallel compression via write_compressed_frame to DataFile

Compare with previous version

Karim Ahmed @ahmedk · 2 years ago

Owner

Tested, LGTM

Philipp Schmidt resolved all threads 2 years ago

resolved all threads

Philipp Schmidt changed milestone to 2 years ago

changed milestone to %3.8.0

Philipp Schmidt @schmidtp · 2 years ago

Author Owner

Thanks for testing and review!

Philipp Schmidt merged 2 years ago

merged

Philipp Schmidt mentioned in commit 2 years ago

mentioned in commit 11aed53d

Please register or sign in to reply

Legal Notice | Contacts