Skip to content

Use new CalCat interface in LPD correction

Thomas Kluyver requested to merge feat/extra-cal-lpd into master

This is not ready to merge, not least because installing we'll need to upgrade to at least Python 3.9 to use EXtra, and that requires changing other dependencies too. But I wanted to give an idea what it looks like.

This now uses the copy in cal_tools to try out the API before we can use EXtra.

Notes:

  • We've previously worked with calibration (meta)data with modules as the outer layer (d['LPD00']['Offset']), but the new API works the other way (d['Offset']['LPD00']). So fitting it into our current code is kind of awkward. We could probably make it more elegant by making more changes, but e.g. changing the format of retrieved-constants in the YAML metadata could break stuff downstream.
  • Handling missing constants and carrying on with the modules & constants available still makes it clunkier. Maybe we could do more to address that, but I'm not sure what right now. 🤷
  • We lose some parallelism loading constants because we can only parallelise over modules within each constant type; the previous implementation could also load different constant types in parallel.
  • My trimming back of the API before we made it public dropped the begin_validity_at field from myMdC, which we save as creation-time in the YAML metadata (although it's not when the constant was created 😕 ).

@schmidtp @ahmedk

Edited by Thomas Kluyver

Merge request reports