ENGINEERING

Auto-Quarantine MR for Persistent Flaky Specs

For tests that stay flaky past a grace period, an agent opens a GitLab merge request that applies a skip/quarantine annotation to the spec, links the tracking issue.

CategoryEngineering
Enginepaperclip
Difficultyadvanced
Triggerschedule
Steps5
Setup~25 min

How it runs

The automated pipeline, trigger to output.

  • TriggerDaily schedule
  • ActionList stale flaky::quarantine issues past grace periodGitLabGitLab
  • LogicLocate failing spec and choose skip annotation
  • ActionEdit test file and open quarantine MRGitLabGitLab
  • OutputAssign code owner and link issue to the MRGitLabGitLab

What it does

Takes flaky tests that have been flagged but unaddressed beyond a grace period and actually proposes the fix: an agent edits the test file to add the appropriate quarantine annotation for your framework, opens a merge request, links the existing tracking issue, and requests review from the spec's code owner.

When to use it

Use it when quarantine issues pile up but nobody gets around to silencing the tests, so the same flaky specs keep poisoning unrelated pipelines. This closes the loop with a reviewable change rather than auto-merging anything.

How it works

  1. 1A daily schedule trigger fires.
  2. 2The agent queries GitLab for `flaky::quarantine` issues older than the grace period (default 5 days).
  3. 3For each, it reads the referenced test file, locates the failing spec, and applies the framework-correct skip annotation with a link back to the issue.
  4. 4It opens a merge request on a new branch with a descriptive title and body.
  5. 5It resolves the code owner from CODEOWNERS and assigns them as reviewer, then comments on the tracking issue with the MR link.

Set it up

What you configure once, before turning it on.

  1. 1
    Connect GitLabRepos, MRs, pipelines, registry.
  2. 2
    Set each agent's modelWe leave models unset so you pick the tier — fast + cheap, or top-quality.
  3. 3
    Tune it to your dataEdit the prompts, filters, and field mappings so it matches how your team works.
  4. 4
    Test, then turn it onRun once against a sample, confirm the output, then enable the trigger.

Run this workflow in your colony.

14-day trial. No DevOps. No Sales call. Provisioned in under a minute.