ENGINEERING

Dependency Canary: Draft PR with Honeycomb Latency Watch

Opens a draft PR for a dependency bump, waits for the canary deploy to take traffic, then compares Honeycomb p95 latency before vs after and posts a verdict to Slack.

CategoryEngineering
Enginesim
Difficultyintermediate
Triggerevent
Steps5
Setup~15 min

How it runs

The automated pipeline, trigger to output.

  • TriggerPR labeled 'dependency' openedGitHubGitHub
  • ActionMark PR draft and add 'canary' labelGitHubGitHub
  • ActionQuery Honeycomb p95 for canary + baseline windowsHoneycomb
  • LogicCompute latency delta vs threshold
  • OutputPost verdict and numbers to SlackSlack

What it does

Turns a routine dependency bump into a measured experiment. When a bot opens a bump PR, this workflow promotes it to a draft canary, watches Honeycomb traces for the affected service, and tells you in Slack whether p95 latency moved.

When to use it

Use it for any auto-generated upgrade PR (Dependabot, Renovate, or a teammate) where you want trace-backed proof the new version is safe before a human reviews and merges. Best for services you already instrument in Honeycomb.

How it works

  1. 1A new pull request labeled `dependency` triggers the run.
  2. 2The workflow ensures the PR is a draft and tags it `canary`.
  3. 3It waits a fixed bake window, then queries Honeycomb for p95 `duration_ms` on the service over the canary period and a matched baseline window.
  4. 4A logic step computes the delta; if p95 rose beyond the threshold it marks the run regressed.
  5. 5It posts the before/after numbers, the verdict, and the PR link to Slack so the reviewer decides with data in hand.

Set it up

What you configure once, before turning it on.

  1. 1
    Connect GitHubRepos, issues, pull requests, actions.
  2. 2
    Connect HoneycombDistributed traces and queries.
  3. 3
    Connect SlackChannels, DMs, threads, mentions.
  4. 4
    Set each agent's modelWe leave models unset so you pick the tier — fast + cheap, or top-quality.
  5. 5
    Tune it to your dataEdit the prompts, filters, and field mappings so it matches how your team works.
  6. 6
    Test, then turn it onRun once against a sample, confirm the output, then enable the trigger.

Run this workflow in your colony.

14-day trial. No DevOps. No Sales call. Provisioned in under a minute.