ENGINEERING

Auto-Unquarantine Tests After a Clean Stability Streak

On a schedule, checks the ledger for quarantined tests that have passed cleanly across enough recent runs, removes them from quarantine.

CategoryEngineering
Enginesim
Difficultyintermediate
Triggerschedule
Steps6
Setup~15 min

How it runs

The automated pipeline, trigger to output.

  • TriggerDaily schedule fires
  • ActionQuery quarantined tests and recent run historyGoogle BigQueryBigQuery
  • LogicSelect tests with clean pass streak over threshold
  • ActionClear quarantine flag in the ledgerGoogle BigQueryBigQuery
  • ActionOpen PR removing the quarantine skipGitHubGitHub
  • OutputClose the deflake issue with a resolution noteLinearLinear

What it does

Quarantine should be temporary. This workflow finds tests that have been quarantined but have since passed cleanly across a configurable number of consecutive runs, marks them stable in the ledger, opens a pull request that re-enables them in the test config, and closes the tracking deflake issue. It prevents the quarantine list from silently growing forever.

When to use it

Use it to reclaim test coverage automatically after fixes land, and to keep your quarantine list honest so it reflects only genuinely unstable tests rather than ones already healed.

How it works

  1. 1A daily schedule trigger fires.
  2. 2The flow queries the BigQuery ledger for quarantined tests and their recent run history.
  3. 3Logic selects tests with an unbroken clean pass streak above the threshold.
  4. 4The ledger marks those tests as stable and clears the quarantine flag.
  5. 5A GitHub pull request is opened removing the quarantine skip from the test config.
  6. 6Each corresponding Linear deflake issue is closed with a resolution note.

Set it up

What you configure once, before turning it on.

  1. 1
    Connect BigQueryDatasets, queries, schemas.
  2. 2
    Connect GitHubRepos, issues, pull requests, actions.
  3. 3
    Connect LinearIssues, projects, cycles, triage.
  4. 4
    Set each agent's modelWe leave models unset so you pick the tier — fast + cheap, or top-quality.
  5. 5
    Tune it to your dataEdit the prompts, filters, and field mappings so it matches how your team works.
  6. 6
    Test, then turn it onRun once against a sample, confirm the output, then enable the trigger.

Run this workflow in your colony.

14-day trial. No DevOps. No Sales call. Provisioned in under a minute.