DEVOPS

Deploy Latency Regression to GitHub Scaling Issue

After a deploy, this workflow compares post-deploy p95 latency against the pre-deploy baseline in Datadog and, on a confirmed regression.

CategoryDevOps
Enginesim
Difficultyintermediate
Triggerwebhook
Steps5
Setup~15 min

How it runs

The automated pipeline, trigger to output.

  • TriggerDeploy webhook received with release idHTTP webhook
  • ActionQuery Datadog post-deploy vs pre-deploy p95 after stabilizationDatadogDatadog
  • LogicBranch on regression exceeding threshold; exit if within budget
  • ActionCompute interim scaling recommendation to hold SLO
  • OutputOpen GitHub issue linking deploy, metrics, and recommendationGitHubGitHub

What it does

Triggered by a deploy webhook, it waits for a stabilization window, then compares post-deploy p95 latency against the pre-deploy baseline in Datadog. If latency regressed beyond the threshold, it opens a GitHub issue that ties the regression to the specific deploy and includes a recommended interim scaling change to absorb the new cost-per-request.

When to use it

Use it to catch deploys that quietly raise per-request latency and force you to over-scale. It gives the owning team a documented, actionable issue instead of a vague alert, with both the code suspect and the capacity workaround in one place.

How it works

  1. 1A deploy webhook triggers the flow with the new release identifier.
  2. 2The flow waits for a stabilization window, then queries Datadog for post-deploy vs. pre-deploy p95.
  3. 3A logic branch checks whether the regression exceeds the threshold; if not, it exits.
  4. 4It computes the interim scaling recommendation needed to hold the SLO at the new latency.
  5. 5It opens a GitHub issue linking the deploy, the regression metrics, and the recommendation.

Set it up

What you configure once, before turning it on.

  1. 1
    Connect HTTP webhookTrigger any URL on agent actions.
  2. 2
    Connect DatadogMetrics, traces, log search.
  3. 3
    Connect GitHubRepos, issues, pull requests, actions.
  4. 4
    Set each agent's modelWe leave models unset so you pick the tier — fast + cheap, or top-quality.
  5. 5
    Tune it to your dataEdit the prompts, filters, and field mappings so it matches how your team works.
  6. 6
    Test, then turn it onRun once against a sample, confirm the output, then enable the trigger.

Run this workflow in your colony.

14-day trial. No DevOps. No Sales call. Provisioned in under a minute.