DEVOPS

Kill the most recent flag rollout when PagerDuty opens an incident

When a high-urgency PagerDuty incident opens, it correlates the incident time with active flag rollouts, disables the most recently changed flag via GitHub.

CategoryDevOps
Enginesim
Difficultyadvanced
Triggerwebhook
Steps6
Setup~25 min

How it runs

The automated pipeline, trigger to output.

  • TriggerPagerDuty webhook on new high-urgency incidentPagerDutyPagerDuty
  • ActionFind recent flag rollouts in incident lookback window via GitHubGitHubGitHub
  • LogicBranch: exactly one correlating rollout?
  • ActionDisable the suspect flag via GitHub commitGitHubGitHub
  • ActionAnnotate PagerDuty incident with the suspect flagPagerDutyPagerDuty
  • OutputNotify responders in SlackSlack

What it does

Treats a fresh production incident as a signal to undo the last thing you changed. It finds the flag that ramped just before the incident, turns it off, and annotates the incident with that hypothesis.

When to use it

Use when flag changes are your most frequent production variable and you want incident response to automatically test the obvious culprit first, shaving minutes off mean-time-to-recovery.

How it works

  1. 1A PagerDuty webhook fires on a new high-urgency incident.
  2. 2A GitHub action reads recent flag-config commits to find rollouts that advanced within the incident's lookback window.
  3. 3A logic branch checks whether exactly one recent rollout correlates; if zero or many, it skips the auto-kill and only annotates.
  4. 4When a single suspect exists, a GitHub commit disables that flag.
  5. 5A PagerDuty note is added to the incident naming the disabled flag and the correlating commit.
  6. 6A Slack message alerts responders that an automatic rollback was attempted.

Set it up

What you configure once, before turning it on.

  1. 1
    Connect PagerDutyIncidents, on-call, escalations.
  2. 2
    Connect GitHubRepos, issues, pull requests, actions.
  3. 3
    Connect SlackChannels, DMs, threads, mentions.
  4. 4
    Set each agent's modelWe leave models unset so you pick the tier — fast + cheap, or top-quality.
  5. 5
    Tune it to your dataEdit the prompts, filters, and field mappings so it matches how your team works.
  6. 6
    Test, then turn it onRun once against a sample, confirm the output, then enable the trigger.

Run this workflow in your colony.

14-day trial. No DevOps. No Sales call. Provisioned in under a minute.