Skip to content
Snippets Groups Projects

[AGIPD][CORRECT] maxtaskperchild multiprocessing pool configuration

Merged [AGIPD][CORRECT] maxtaskperchild multiprocessing pool configuration
feat/maxtaskperchild into master
All threads resolved!
Merged Karim Ahmed requested to merge feat/maxtaskperchild into master
All threads resolved!

Add maxtasksperchild configuration to reduce number of tasks per worker and avoid possible memory leaks

Description

While debugging the Out of memory issue while correcting very long AGIPD runs. @kluyvert pointed to an option to specify the number of tasks a worker can handle before it gets replaced with a new worker.

How Has This Been Tested?

Relevant Documents (optional)

Types of changes

  • New feature (non-breaking change which adds functionality)

Checklist:

Reviewers

@schmidtp @kluyvert

Edited by Karim Ahmed

Merge request reports

Loading
Loading

Activity

Filter activity
  • Approvals
  • Assignees & reviewers
  • Comments (from bots)
  • Comments (from users)
  • Commits & branches
  • Edits
  • Labels
  • Lock status
  • Mentions
  • Merge request status
  • Tracking
  • Karim Ahmed added 1 commit

    added 1 commit

    • 4ebf733f - maxtaskperchild multiprocessing pool configuration

    Compare with previous version

  • Karim Ahmed resolved all threads

    resolved all threads

  • Thomas Kluyver
    Thomas Kluyver @kluyvert ·
    Owner

    LGTM

  • Philipp Schmidt changed milestone to %3.5.5

    changed milestone to %3.5.5

  • Philipp Schmidt
    Philipp Schmidt @schmidtp ·
    Owner

    Thanks, LGTM

  • Karim Ahmed
    Karim Ahmed @ahmedk ·
    Author Owner

    Thanks for the review.

  • Karim Ahmed merged

    merged

  • Karim Ahmed mentioned in commit 56ec9297

    mentioned in commit 56ec9297

    • Please register or sign in to reply

    Legal Notice | Contacts