test/e2e tests

Description

Adding end to end tests for the calibration pipeline.

Approach is to:

0. Modify webservice.yaml to be suitable for a test environment
1. Start up a(n easily killable and ideally trackable by codecov) instance of the webservice server
2. Start up a ZMQ client with a REQ socket that can send calibration requests and receive replies like MyMdC
3. Mock calls to slurm (and other stuff) to pretend to be running on Maxwell, but just execute everything directly and return sensible replies (e.g. when 'slurm' submits a job, return a fake job id, check that it is logged)
4. Select a small subset of data from CALLAB runs to be used for 'quick' tests as part of a PR
5. Set up nightly runs which run on the entire dataset, and run after a merge is done

How Has This Been Tested?

It is a test

Relevant Documents (optional)

N/A

Types of changes

• Test (additional or refactored tests)

Checklist:

• My code follows the code style of this project.
• My change requires a change to the documentation.
• I have updated the documentation accordingly.

Reviewers

Very WIP so none, I'll write a comment when this is closer to completion

Notes

• resolved - Still don't have much experience with asyncio, and even less with testing background servers. Looks like a session scoped fixture which starts a process that executes webservice.webservice.main, yields, and then terminates the process afterwards should work. I'm not sure if this will be tracked by codecov though, that isn't too big a priority for this but it would be nice

webservice/webservice.py

@@ -791,6 +791,9 @@ class ActionsServer:
             )
 
         except Exception as e:
+            #  TODO: error log occurs twice, once here and once in the update
+            #  mdc status call, a bit confusing to figure out which line causes
+            #  it
             msg = Errors.JOB_LAUNCH_FAILED.format('correct', e)
             logging.error(msg, exc_info=e)
             asyncio.ensure_future(