ENGINEERING

Quarantine Dependabot PRs That Break CI and Triage by Failure Type

When a Dependabot PR's CI run completes, it inspects the result and, for failures, classifies them as flaky-retry, breaking-change, or peer-conflict.

CategoryEngineering
Enginesim
Difficultyadvanced
Triggerwebhook
Steps6
Setup~25 min

How it runs

The automated pipeline, trigger to output.

  • TriggerCheck suite completes on Dependabot PRGitHubGitHub
  • LogicKeep only failed CI runs
  • ActionFetch failing job logs and diffGitHubGitHub
  • LogicClassify failure: flake / breaking / peer-conflict
  • ActionApply quarantine label, re-run flakesGitHubGitHub
  • OutputPost triage verdict to SlackSlack

What it does

A red check on a Dependabot PR could mean a flaky test, a genuine breaking change, or a peer-dependency conflict — and each needs a different response. This workflow reads the failed CI run, classifies the failure from the logs, quarantines the PR with a clear label, and tells reviewers exactly what kind of problem they're looking at before anyone opens the PR.

When to use it

Run it when dependency-bump PRs frequently fail CI and reviewers waste time opening each one just to discover whether it's a flake or a real incompatibility.

How it works

  1. 1A GitHub webhook fires when a check suite completes on a Dependabot PR.
  2. 2A filter keeps only failed runs and ignores green ones.
  3. 3GitHub fetches the failing job logs and the dependency diff.
  4. 4A classifier matches log signatures to flaky-retry, breaking-change, or peer-conflict categories.
  5. 5A branch applies the matching quarantine label and, for flakes, requests a re-run.
  6. 6A triage note with the verdict and next action posts to Slack.

Set it up

What you configure once, before turning it on.

  1. 1
    Connect GitHubRepos, issues, pull requests, actions.
  2. 2
    Connect SlackChannels, DMs, threads, mentions.
  3. 3
    Set each agent's modelWe leave models unset so you pick the tier — fast + cheap, or top-quality.
  4. 4
    Tune it to your dataEdit the prompts, filters, and field mappings so it matches how your team works.
  5. 5
    Test, then turn it onRun once against a sample, confirm the output, then enable the trigger.

Run this workflow in your colony.

14-day trial. No DevOps. No Sales call. Provisioned in under a minute.