Support for saving metadata fragments & merging into calibration_metadata.yml

Description

We want to remove the duplication of looking up calibration constants in both a pre- notebook and the main correction notebook. If we adopt CalParrot as planned (!767 (merged)), the only remaining missing bit will be writing the constant details to the calibration_metadata.yml file. The difficulty in doing that from the main notebooks is that they run in parallel - if two happened to write the file at the same time, it could get corrupted.

I see two possible approaches for dealing with this:

1. This MR: Write metadata to fragment files, and then merge them back into calibration_metadata.yml in the finalize step. So we'd temporarily have one extra file per notebook.
2. Branch feat/update-lock-metadata-yml: provide a way to read-update-write a file while holding a lock on it, so if two notebooks tried to do this at the same time, one would be forced to wait to get the lock. However, I'd want to do some more work to allow for a timeout on acquiring the lock, which is unfortunately not as simple to do as I'd like.

One remaining concern with either option: if different copies of the notebook should find different constants - e.g. if a bug means one gets a CalParrot stored reply and the other gets a new reply from CalCat - then what is recorded in calibration_metadata.yml will be random. Perhaps when merging, we should warn if there are conflicting values in the metadata?

How Has This Been Tested?

TBD

Types of changes

• New feature (non-breaking change which adds functionality)

Checklist:

• My code follows the code style of this project.

Reviewers

@ahmedk @schmidtp

src/cal_tools/tools.py

@@ -813,13 +813,23 @@ def module_index_to_qm(index: int, total_modules: int = 16):
 
 
 def recursive_update(target: dict, source: dict):
+    """Recursively merge source into target, checking for conflicts
+
+    Conflicting entries will not be copied to target. Returns True if any
+    conflicts were found.
+    """
+    conflict = False
     for k, v2 in source.items():
         v1 = target.get(k, None)
         if isinstance(v1, dict) and isinstance(v2, dict):
-            recursive_update(v1, v2)
+            conflict = recursive_update(v1, v2) or conflict
+        elif (v1 is not None) and (v1 != v2):
+            conflict = True
         else:
             target[k] = v2
 
+    return conflict
+
 class CalibrationMetadata(dict):
     """Convenience class: dictionary stored in metadata YAML file
 
@@ -868,14 +878,19 @@ class CalibrationMetadata(dict):
     def gather_fragments(self):
         """Merge in fragments saved by add_fragment(), then delete them"""
         frag_files = list(self._yaml_fn.parent.glob('metadata_frag_*.yml'))
+        to_delete = []
         for fn in frag_files:
             with fn.open("r") as fd:
                 data = yaml.safe_load(fd)
-                recursive_update(self, data)
+                if recursive_update(self, data):
+                    print(f"{fn} contained conflicting metadata. "
+                          f"This file will be left for debugging")
+                else:
+                    to_delete.append(fn)
 
         self.save()
 
-        for fn in frag_files:
+        for fn in to_delete:
             fn.unlink()