ENGINEERING

Nightly p99 query regression digest tied to the day's deploys

Each night this compares today's slowest queries in Honeycomb against yesterday's baseline and posts a Slack digest that pairs each regression with the deploys most likely…

CategoryEngineering
Enginesim
Difficultyintermediate
Triggerschedule
Steps5
Setup~15 min

How it runs

The automated pipeline, trigger to output.

  • TriggerNightly schedule fires
  • ActionPull per-query p99 and baseline from HoneycombHoneycomb
  • LogicFlag queries that regressed past the margin
  • ActionMatch GitHub deploys to each regression windowGitHubGitHub
  • OutputPost ranked regression digest to SlackSlack

What it does

Produces a once-a-day scorecard of which database queries got slower and which deploys probably caused it. It diffs today's per-query p99 against the prior baseline and attaches the deploys that shipped during each query's regression.

When to use it

Use it when you want a steady morning pulse on query health rather than only reacting to alerts — catching the slow creep that never trips a single-threshold trigger but compounds over a week.

How it works

  1. 1A scheduled run fires every night after the deploy freeze.
  2. 2It queries Honeycomb for per-query p99 over the last 24 hours and the prior 7-day baseline.
  3. 3A logic step flags queries whose p99 rose beyond the regression margin.
  4. 4For each flagged query it pulls the GitHub deploys that landed during the rise.
  5. 5It assembles a ranked digest pairing each regression with its candidate deploys.
  6. 6The digest posts to the engineering Slack channel with trace and commit links.

Set it up

What you configure once, before turning it on.

  1. 1
    Connect HoneycombDistributed traces and queries.
  2. 2
    Connect GitHubRepos, issues, pull requests, actions.
  3. 3
    Connect SlackChannels, DMs, threads, mentions.
  4. 4
    Set each agent's modelWe leave models unset so you pick the tier — fast + cheap, or top-quality.
  5. 5
    Tune it to your dataEdit the prompts, filters, and field mappings so it matches how your team works.
  6. 6
    Test, then turn it onRun once against a sample, confirm the output, then enable the trigger.

Run this workflow in your colony.

14-day trial. No DevOps. No Sales call. Provisioned in under a minute.