test/e2e tests

Description

Adding end to end tests for the calibration pipeline.

Approach is to:

0. Modify webservice.yaml to be suitable for a test environment
1. Start up a(n easily killable and ideally trackable by codecov) instance of the webservice server
2. Start up a ZMQ client with a REQ socket that can send calibration requests and receive replies like MyMdC
3. Mock calls to slurm (and other stuff) to pretend to be running on Maxwell, but just execute everything directly and return sensible replies (e.g. when 'slurm' submits a job, return a fake job id, check that it is logged)
4. Select a small subset of data from CALLAB runs to be used for 'quick' tests as part of a PR
5. Set up nightly runs which run on the entire dataset, and run after a merge is done

How Has This Been Tested?

It is a test

Relevant Documents (optional)

N/A

Types of changes

• Test (additional or refactored tests)

Checklist:

• My code follows the code style of this project.
• My change requires a change to the documentation.
• I have updated the documentation accordingly.

Reviewers

Very WIP so none, I'll write a comment when this is closer to completion

Notes

• resolved - Still don't have much experience with asyncio, and even less with testing background servers. Looks like a session scoped fixture which starts a process that executes webservice.webservice.main, yields, and then terminates the process afterwards should work. I'm not sure if this will be tracked by codecov though, that isn't too big a priority for this but it would be nice

added In Progress label

added 25 commits

• 3e3553c7...fa2d8cad - 21 commits from branch master
• 2c130c94 - Organise tests by module
• feadc06e - Initial conftest
• 119ac5c8 - Use dynaconf for configuration in tests
• 9eb74421 - Save test progress

Compare with previous version

Expect a minor delay as I accidentally deleted my work and had not done a push in ages

added 2 commits

• baa72f2a - Additional notes on configuration
• a042544a - Update gitignore

Compare with previous version

added 1 commit

• a6f47ae4 - Move MockProposal to top level conftest

Compare with previous version

added 1 commit

• f38d2f6a - Rename instrument attribute, add detector

Compare with previous version

added 1 commit

• ccc343cb - Fix accidentally swapped variables

Compare with previous version

added 1 commit

• d8361336 - Add testpath command mocking

Compare with previous version

added 1 commit

• 1dcd65af - Commit debugging changes - remember to remove before PR

Compare with previous version

added 1 commit

• e4978459 - Write tests for tests

Compare with previous version

added 1 commit

• bc38d55b - Minor changes to testing config

Compare with previous version

added 1 commit

• bc38d55b - Minor changes to testing config

Compare with previous version

added 1 commit

• 3e7e2482 - Remove loggers before setting up webservice log

Compare with previous version

added 1 commit

• 0ce021c3 - Set run directory attribute

Compare with previous version

added 7 commits

• c52c711d - Mock mymdc update run api method
• 135220c6 - (try to) Mock python calls
• 5d72399f - Add todo on exception handling and logging
• 1e3297a7 - More logs to try and figure out what's happening!
• 308acb19 - Expand mocked commands
• cf580e59 - Fix setattr call, use pass mocked python call to
• 83a6df45 - Replace single with double quotes, use block chomp

Compare with previous version

@kluyvert should, or could, something like this (a way to create an entire mock proposal directory) go into extra-data? Seems like it might be useful for future tests in extra-data, and it would definitely be useful to have a standard approach here both for our own testing and as a way to let users create their own mock data before a beamtime so they can explore how to develop their analysis tools

Edit: also, this class has only been moved from another conftest file

Yup, could do. There are a couple of places in EXtra-data where we test with a proposal directory structure. And it probably makes sense for this to live alongside the machinery for creating mock data.