Revised CalCat API

This is based on our discussion last week, but it's still a WIP. In particular, I haven't implemented actually loading the constants yet.

I've tweaked the design slightly: rather than having CalibrationData subclasses for each detector type, you use CalibrationData.from_conditions() with a detector specific object like AGIPDConditions. Jump to the tests to see what this looks like.

Questions:

• How to deal with missing modules?
  • A module may be missing for one type of constant but not for another
  • How do we even tell if modules are missing - should we query the PDUs so we know how many modules to expect?
• How to deal with missing calibration types (e.g. SlopesPC)
  • Especially if one group (e.g. illuminated) not found - this currently raises an error, even if we have other constants we could return
• Naming:
  • 'Calibrations' vs 'calibration' types vs 'constant types' (e.g. Offset, SlopesPC
  • 'Modules' vs 'aggregators'
• How many of SingleConstantVersion fields do we actually want?
• Should .select_modules() and .select_calibrations() take *args or a list?

cc @schmidtp @ahmedk @hammerd

src/cal_tools/calcat_interface2.py

@@ -2,7 +2,7 @@ import json
 import re
 from collections.abc import Mapping
 from dataclasses import dataclass
-from datetime import datetime, date, time, timezone
+from datetime import date, datetime, time, timezone
 from functools import lru_cache
 from pathlib import Path
 from typing import Dict, List, Optional, Union
@@ -341,7 +341,7 @@ class CalibrationData(Mapping):
 
     def __init__(self, constant_groups, module_details):
         self.constant_groups = {
-            const_type: ModulesConstantVersions(d)
+            const_type: ModulesConstantVersions(d, module_details)
             for const_type, d in constant_groups.items()
         }
         self.module_details = module_details
@@ -448,15 +448,15 @@ class CalibrationData(Mapping):
         res = client.get("calibration_constant_versions", params)
 
         d = {}
-        aggregators = set()
+        pdus = []
 
         for ccv in res:
-            aggr = ccv["physical_detector_unit"]["karabo_da"]
-            aggregators.add(aggr)
+            pdus.append(ccv["physical_detector_unit"])
             cal_type = calibration_name(ccv["calibration_constant"]["calibration_id"])
+            aggr = ccv["physical_detector_unit"]["karabo_da"]
             d.setdefault(cal_type, {})[aggr] = SingleConstantVersion.from_response(ccv)
 
-        return cls(d, sorted(aggregators))
+        return cls(d, sorted(pdus, key=lambda d: d["karabo_da"]))
 
     def __getitem__(self, key) -> ModulesConstantVersions:
         return self.constant_groups[key]