DEVOPS

Auto-unquarantine tests that have been stable for 14 days

Weekly, checks Datadog for quarantined tests that have passed every run for 14 consecutive days, removes them from the quarantine manifest via a GitHub PR.

CategoryDevOps
Enginesim
Difficultyintermediate
Triggerschedule
Steps6
Setup~15 min

How it runs

The automated pipeline, trigger to output.

  • TriggerWeekly schedule
  • ActionRead quarantine manifest from GitHubGitHubGitHub
  • ActionQuery Datadog for 14-day pass/fail historyDatadogDatadog
  • LogicClean streak and minimum run count met?
  • ActionOpen GitHub PR removing stable testsGitHubGitHub
  • OutputClose matching Linear issues as resolvedLinearLinear

What it does

Closes the loop on quarantine. On a weekly schedule it reviews every test on the quarantine list, confirms via Datadog that it has had zero failures across a 14-day window, then opens a GitHub pull request removing it from the manifest and closes the corresponding Linear issue. Quarantine stops being a graveyard and becomes a temporary holding pen.

When to use it

Use this when fixed-but-still-quarantined tests accumulate because nobody remembers to re-enable them. It restores real coverage automatically once a test has proven itself stable.

How it works

  1. 1A weekly schedule fires.
  2. 2The flow reads the current quarantine manifest from GitHub.
  3. 3For each test, it queries Datadog for the pass/fail history over the last 14 days.
  4. 4A filter keeps only tests with a clean streak and a minimum run count.
  5. 5It opens a GitHub PR removing those tests from the manifest, citing the stability evidence.
  6. 6The final step closes each matching Linear issue with a resolution note and links the cleanup PR.

Set it up

What you configure once, before turning it on.

  1. 1
    Connect GitHubRepos, issues, pull requests, actions.
  2. 2
    Connect DatadogMetrics, traces, log search.
  3. 3
    Connect LinearIssues, projects, cycles, triage.
  4. 4
    Set each agent's modelWe leave models unset so you pick the tier — fast + cheap, or top-quality.
  5. 5
    Tune it to your dataEdit the prompts, filters, and field mappings so it matches how your team works.
  6. 6
    Test, then turn it onRun once against a sample, confirm the output, then enable the trigger.

Run this workflow in your colony.

14-day trial. No DevOps. No Sales call. Provisioned in under a minute.