ENGINEERING

Re-enable stabilized tests and page on quarantine release-blocks

Daily, checks quarantined tests against recent CI history: opens a PR to un-quarantine any test that has been green for a sustained window.

CategoryEngineering
Enginesim
Difficultyadvanced
Triggerschedule
Steps6
Setup~25 min

How it runs

The automated pipeline, trigger to output.

  • TriggerDaily schedule fires
  • ActionRead quarantined tests and recent results from GitHubGitHubGitHub
  • LogicSplit into stable-to-re-enable vs still-flaky
  • ActionOpen PR removing quarantine for stable testsGitHubGitHub
  • ActionPage on-call for release-critical still-quarantined testsPagerDutyPagerDuty
  • OutputPost re-enable and escalation summary to SlackSlack

What it does

Closes the loop on quarantine so tests do not stay disabled forever. It reviews the set of currently quarantined tests, measures their recent stability from CI run history, and re-enables the ones that have proven reliable again. It also catches the dangerous case where a quarantined test guards a release-critical path and escalates it.

When to use it

Use it once you have an automated or manual quarantine process and need to prevent permanent coverage gaps. Essential for teams with release gates that quarantine could silently weaken.

How it works

  1. 1A daily schedule fires the workflow.
  2. 2It reads the quarantined-test list and their recent results from GitHub run history.
  3. 3A logic step splits tests into 'stable enough to re-enable' versus 'still flaky' using a sustained-green window.
  4. 4For stable tests it opens a GitHub PR removing the quarantine annotation.
  5. 5For still-quarantined tests on a release-critical path it pages on-call via PagerDuty.
  6. 6It posts the day's re-enable and escalation summary to Slack.

Set it up

What you configure once, before turning it on.

  1. 1
    Connect GitHubRepos, issues, pull requests, actions.
  2. 2
    Connect PagerDutyIncidents, on-call, escalations.
  3. 3
    Connect SlackChannels, DMs, threads, mentions.
  4. 4
    Set each agent's modelWe leave models unset so you pick the tier — fast + cheap, or top-quality.
  5. 5
    Tune it to your dataEdit the prompts, filters, and field mappings so it matches how your team works.
  6. 6
    Test, then turn it onRun once against a sample, confirm the output, then enable the trigger.

Run this workflow in your colony.

14-day trial. No DevOps. No Sales call. Provisioned in under a minute.