Revised CalCat API

This is based on our discussion last week, but it's still a WIP. In particular, I haven't implemented actually loading the constants yet.

I've tweaked the design slightly: rather than having CalibrationData subclasses for each detector type, you use CalibrationData.from_conditions() with a detector specific object like AGIPDConditions. Jump to the tests to see what this looks like.

Questions:

• How to deal with missing modules?
  • A module may be missing for one type of constant but not for another
  • How do we even tell if modules are missing - should we query the PDUs so we know how many modules to expect?
• How to deal with missing calibration types (e.g. SlopesPC)
  • Especially if one group (e.g. illuminated) not found - this currently raises an error, even if we have other constants we could return
• Naming:
  • 'Calibrations' vs 'calibration' types vs 'constant types' (e.g. Offset, SlopesPC
  • 'Modules' vs 'aggregators'
• How many of SingleConstantVersion fields do we actually want?
• Should .select_modules() and .select_calibrations() take *args or a list?

cc @schmidtp @ahmedk @hammerd

src/cal_tools/calcat_interface2.py

@@ -340,7 +340,7 @@ class MultiModuleConstant(Mapping):
         for m in self.module_details:
             names = (m["module_number"], m["virtual_device_name"], m["physical_name"])
             if key in names and m["karabo_da"] in self.constants:
-                candidate_kdas.add([m["karabo_da"]])
+                candidate_kdas.add(m["karabo_da"])
 
         if not candidate_kdas:
             raise KeyError(key)
@@ -550,6 +550,10 @@ class CalibrationData(Mapping):
             kda = pdu["karabo_da"] = pdu.pop("karabo_da_at_ccv_begin_at")
             det_id = pdu["detector_id"] = pdu.pop("detector_id_at_ccv_begin_at")
             pdu["virtual_device_name"] = pdu.pop("virtual_device_name_at_ccv_begin_at")
+            if pdu.get("module_number_at_ccv_begin_at") is not None:
+                pdu["module_number"] = pdu.pop("module_number_at_ccv_begin_at")
+            else:
+                pdu["module_number"] = int(re.findall(r"\d+", kda)[-1])
 
             det_ids.add(det_id)
             if kda in pdus: