DEVOPS

Freeze deploys and page on-call when error budget burns fast

When Datadog fires a fast-burn SLO alert via webhook, this workflow flips a deploy-freeze flag, comments the freeze on open release PRs in GitHub.

CategoryDevOps
Enginesim
Difficultyadvanced
Triggerwebhook
Steps5
Setup~25 min

How it runs

The automated pipeline, trigger to output.

  • TriggerDatadog fast-burn webhookDatadogDatadog
  • LogicParse service, burn multiplier, budget
  • ActionSet deploy-freeze flag in PostgresPostgreSQLPostgres
  • ActionComment freeze on open release PRsGitHubGitHub
  • OutputPage service owner in PagerDutyPagerDutyPagerDuty

What it does

Reacts to a fast-burn alert by automatically declaring a deploy freeze for the affected service. It records the freeze, signals it on in-flight release pull requests so nobody merges into a burning service, and escalates to the right responder.

When to use it

Use it when a sudden spike can blow your monthly budget in hours and you need an enforced freeze rather than a polite Slack note. Best for services with a strict error-budget policy and clear ownership.

How it works

  1. 1A Datadog fast-burn SLO monitor sends a webhook to the flow.
  2. 2A logic step parses the alert to extract the service, current burn multiplier, and remaining budget.
  3. 3The flow sets the per-service deploy-freeze flag in a Postgres control table other gates read.
  4. 4It posts a freeze comment on open release-labeled GitHub PRs for that service.
  5. 5It opens a PagerDuty incident routed to the service owner with the budget figures attached.

Set it up

What you configure once, before turning it on.

  1. 1
    Connect DatadogMetrics, traces, log search.
  2. 2
    Connect GitHubRepos, issues, pull requests, actions.
  3. 3
    Connect PagerDutyIncidents, on-call, escalations.
  4. 4
    Connect PostgresAny Postgres URL — query, write, migrate.
  5. 5
    Set each agent's modelWe leave models unset so you pick the tier — fast + cheap, or top-quality.
  6. 6
    Tune it to your dataEdit the prompts, filters, and field mappings so it matches how your team works.
  7. 7
    Test, then turn it onRun once against a sample, confirm the output, then enable the trigger.

Run this workflow in your colony.

14-day trial. No DevOps. No Sales call. Provisioned in under a minute.