ENGINEERING

Auto-Skip Flaky Test via Quarantine MR

On a confirmed flaky test, opens a GitLab merge request that adds a skip annotation referencing a tracking issue, so the test stops blocking the pipeline immediately.

CategoryEngineering
Enginesim
Difficultyadvanced
Triggerevent
Steps6
Setup~25 min

How it runs

The automated pipeline, trigger to output.

  • TriggerIssue labeled auto-skip (GitLab webhook)GitLabGitLab
  • ActionRead issue for test path and idGitLabGitLab
  • LogicSkip if test already quarantined
  • ActionCommit skip annotation to a new branchGitLabGitLab
  • ActionOpen quarantine merge requestGitLabGitLab
  • OutputComment MR link on tracking issueGitLabGitLab

What it does

This workflow doesn't just file paperwork — it removes the flaky test from the critical path. When a flaky test is confirmed, it opens a GitLab merge request that adds a skip/quarantine annotation to the test, linked back to a tracking issue, so the next pipeline run is green while the root cause stays visible.

When to use it

Reach for this when a single flaky test is repeatedly blocking merges and you want it auto-skipped within minutes, not after a manual triage cycle. The MR keeps a human in the loop to approve the skip.

How it works

  1. 1A GitLab webhook fires when a quarantine tracking issue is labeled `auto-skip`.
  2. 2The flow reads the issue to extract the test path and identifier.
  3. 3A logic step confirms the test isn't already skipped to avoid duplicate MRs.
  4. 4It creates a branch and commits a skip annotation (e.g. `@pytest.mark.skip` / `it.skip`) referencing the issue number.
  5. 5It opens a merge request with the change, requesting review from the test's last author.
  6. 6It comments on the original issue linking the quarantine MR.

Set it up

What you configure once, before turning it on.

  1. 1
    Connect GitLabRepos, MRs, pipelines, registry.
  2. 2
    Set each agent's modelWe leave models unset so you pick the tier — fast + cheap, or top-quality.
  3. 3
    Tune it to your dataEdit the prompts, filters, and field mappings so it matches how your team works.
  4. 4
    Test, then turn it onRun once against a sample, confirm the output, then enable the trigger.

Run this workflow in your colony.

14-day trial. No DevOps. No Sales call. Provisioned in under a minute.