ENGINEERING

Nightly re-test and rehabilitate quarantined specs

Each night, re-runs every quarantined spec multiple times in isolation; specs that pass a configured number of consecutive runs get their quarantine removed, a cleanup PR opened.

CategoryEngineering
Enginesim
Difficultyadvanced
Triggerschedule
Steps5
Setup~25 min

How it runs

The automated pipeline, trigger to output.

  • TriggerSchedule: nightly re-test run
  • ActionDispatch isolated re-runs of quarantined specsGitHubGitHub
  • ActionRecord consecutive-pass streaksPostgreSQLPostgres
  • LogicSelect specs meeting clean-streak threshold
  • OutputOpen cleanup PR and close quarantine issueGitHubGitHub

What it does

Prevents the quarantine list from becoming a graveyard. It actively verifies whether quarantined specs have stabilized — either because the underlying flake was fixed or the environment changed — and automatically promotes proven-stable specs back into the active suite.

When to use it

Use this alongside any quarantine program so isolated tests are continuously re-evaluated instead of forgotten. It is the rehabilitation half of the workflow.

How it works

  1. 1A nightly schedule trigger fires.
  2. 2The flow reads the current quarantine list from Postgres and dispatches an isolated re-run of each spec via the GitHub Actions API, repeated several times.
  3. 3It records the streak of consecutive passes per spec back to the history table.
  4. 4A logic branch selects specs that hit the required clean-streak threshold.
  5. 5For each promoted spec it opens a cleanup PR removing the skip annotation and closes the associated quarantine issue with the passing evidence.

Set it up

What you configure once, before turning it on.

  1. 1
    Connect GitHubRepos, issues, pull requests, actions.
  2. 2
    Connect PostgresAny Postgres URL — query, write, migrate.
  3. 3
    Set each agent's modelWe leave models unset so you pick the tier — fast + cheap, or top-quality.
  4. 4
    Tune it to your dataEdit the prompts, filters, and field mappings so it matches how your team works.
  5. 5
    Test, then turn it onRun once against a sample, confirm the output, then enable the trigger.

Run this workflow in your colony.

14-day trial. No DevOps. No Sales call. Provisioned in under a minute.