Skip to content

Draft: Feat/sentry integration

Robert Rosca requested to merge feat/sentry-integration into main

Adds sentry tracking with some extra stuff to send more information than the bare minimum.

Most minimalistic integration would be doing:

sentry_sdk.init(dsn="https://0b5bdb7568e3e910af144dfb3c324dac@sentry.exfldadev01.desy.de/6")

There's more than that in this PR to add more context and data to what sentry sends:

  • dist set from git status - includes info on the current commit hash and if repo is dirty or not
    • very much not required
    • might also not make sense when deployed
  • functions_to_trace added programatically
    • to get more span info you can add @sentry_sdk.trace to functions/methods
    • alternative to this is providing a list of the fully qualified name of the thing you want to trace
    • doing this by making a list of some classes in calparrot, finding their methods, and adding those to be traced
  • slurm job id tag - we can get env vars and explicitly set them as tags for better searching/visibility
    • could be expanded on
    • e.g. adding proposal number, run number, whatever, as tags
  • setting span data for cache operations
    • by creating a span called cache.put/get and setting data with a few specific keys you tell sentry what this span is for
    • enables a special cache page view in the sentry interface
    • shows statistics like:
      • cache hit/misses
      • cache size
      • read/write times
    • just added this to play with, of all the things probably the most optional

Todo:

  • set environment automatically
    • it's fixed to "testing" right now
    • should be "production" when running on xcal
    • I guess "testing" when on xcaltst
    • and maybe "development" otherwise (e.g. us running it)?
  • set port correctly for cache data
    • if we keep the feature "network.peer.port" should be set to the actual port
    • lazily I hard coded it to 9000
  • enable by default?
    • added a new --sentry flag to the CLI to enable integration
    • might make more sense to use environment to enable/disable conditionally?
      • production/testing always enabled unless --no-sentry passed
      • otherwise disabled unless --sentry passed
    • or presence of DSN as env var?
      • remove hard-coded DSN
      • enable if DSN env var present
      • disable otherwise

Merge request reports

Loading