DEVOPS

Comment quarantine decision on PRs touching flaky tests

When a pull request modifies a test file that is currently quarantined, posts a PR comment explaining the test is flaky, links the tracking issue.

CategoryDevOps
Enginesim
Difficultyintermediate
Triggerevent
Steps5
Setup~15 min

How it runs

The automated pipeline, trigger to output.

  • TriggerGitHub pull_request opened or synchronizedGitHubGitHub
  • ActionDiff changed files against quarantine manifestGitHubGitHub
  • LogicQuarantined test re-enabled without approval label?
  • ActionComment with flake history and Linear issue linkGitHubGitHub
  • OutputSet gating commit status on the PRGitHubGitHub

What it does

Guards the quarantine list at review time. When a pull request changes a test file that appears on the active quarantine manifest, the flow comments on the PR with the test's flake history and tracking issue, and blocks accidental un-quarantining unless a maintainer explicitly approves it with a label.

When to use it

Use this when developers keep re-enabling known-flaky tests in unrelated PRs, reintroducing red builds. It surfaces the quarantine context exactly where the change happens and enforces a deliberate decision.

How it works

  1. 1A GitHub `pull_request` synchronize event fires when a PR is opened or updated.
  2. 2The flow diffs the PR's changed files against the quarantine manifest.
  3. 3A branch checks whether any quarantined test was touched, and whether the diff re-enables it.
  4. 4If a quarantined test is being re-enabled without the `approve-unquarantine` label, it requests changes via a PR comment linking the Linear tracking issue.
  5. 5If the change is safe, it posts an informational comment with the test's flake history.
  6. 6The final step sets a commit status so merge is gated until the decision is acknowledged.

Set it up

What you configure once, before turning it on.

  1. 1
    Connect GitHubRepos, issues, pull requests, actions.
  2. 2
    Connect LinearIssues, projects, cycles, triage.
  3. 3
    Set each agent's modelWe leave models unset so you pick the tier — fast + cheap, or top-quality.
  4. 4
    Tune it to your dataEdit the prompts, filters, and field mappings so it matches how your team works.
  5. 5
    Test, then turn it onRun once against a sample, confirm the output, then enable the trigger.

Run this workflow in your colony.

14-day trial. No DevOps. No Sales call. Provisioned in under a minute.