Skip to content
Snippets Groups Projects

Create the webservice's MyMDC client in a non-blocking thread

Merged Cyril Danilevski requested to merge fix/webservice_mdc_client_init into master
1 unresolved thread

Description

This fixes blocking the event loop by creating the MyMDC client in another thread.

How Has This Been Tested?

This was tested on xcaltst@max-exfl017 to ensure that the web service starts correctly, the client is correctly created, and that jobs can be submitted as usual.

Types of changes

  • Bug fix (non-breaking change which fixes an issue)

Checklist:

  • My code follows the code style of this project.
  • My change requires a change to the documentation.
  • I have updated the documentation accordingly.

Reviewers

@ahmedk @hammerd @roscar

Merge request reports

Loading
Loading

Activity

Filter activity
  • Approvals
  • Assignees & reviewers
  • Comments (from bots)
  • Comments (from users)
  • Commits & branches
  • Edits
  • Labels
  • Lock status
  • Mentions
  • Merge request status
  • Tracking
60 60 """
61 # FIXME: this blocks the even loop, should use asyncio.Task
62 # FIXME: calls to this coro should be shielded
63 61 # TODO: could the client be a global? This would recuce passing it around
64 mdconf = config['metadata-client']
65 client_conn = MetadataClient(client_id=mdconf['user-id'],
66 client_secret=mdconf['user-secret'],
67 user_email=mdconf['user-email'],
68 token_url=mdconf['token-url'],
69 refresh_url=mdconf['refresh-url'],
70 auth_url=mdconf['auth-url'],
71 scope=mdconf['scope'],
72 base_api_url=mdconf['base-api-url'])
73 return client_conn
62
63 # During MetadataClient initialisation, the object does requests to MyMDC
  • Very nice asyncio improvement. I had a very tiny comment that you can ignore if you like.

    LGTM

  • I understand nothing. Anybody got a good asyncio tutorial :wink:

  • added 1 commit

    • fb184ff1 - Apply suggestion to webservice/webservice.py

    Compare with previous version

  • added 1 commit

    • 1ddee74a - Apply suggestion to webservice/webservice.py

    Compare with previous version

  • Technically this looks good to me, but as we discussed in chat, I'm not sure about the direction. We have quite a lot of blocking calls and not a lot of concurrency, especially at initialisation (just two tasks). Wrapping everything in thread executors to make it async seems a bit ugly when it could be simple blocking calls in a couple of threads.

    The bit I'm not sure about is the secondary tasks we create on requests to wait for migration to complete. Maybe it's worth using asyncio for those.

  • Thanks!
    I'll take these points into consideration, and come up with a proposal for refactoring the webservice.

  • mentioned in commit e16be648

  • Please register or sign in to reply
    Loading