ENGINEERING
Quarantine Flaky CI Tests and File a Linear Issue
When a GitHub Actions run fails, detects tests that pass on retry, tags them with a quarantine annotation.
How it runs
The automated pipeline, trigger to output.
- TriggerGitHub Actions workflow run completesGitHub
- LogicDetect tests that failed attempt 1 but passed on retry
- LogicExit if no retry-masked flake found
- ActionLabel PR flaky-quarantine and comment with run linkGitHub
- OutputCreate Linear issue tagged flaky-test with ownerLinear
What it does
Catches the specific failure pattern where a test fails on the first attempt of a CI run but passes on automatic retry — the signature of a flaky test. It marks the offending test as quarantined and files a tracking issue in Linear so the flake gets owned and fixed rather than ignored.
When to use it
Use it when your team relies on CI retries to stay green and flaky tests are slipping through unnoticed. It turns every retry-masked flake into a visible, assigned work item.
How it works
- 1A GitHub Actions workflow run completes and fires a webhook with the conclusion and job attempts.
- 2A logic step inspects the job attempts: keep only tests that failed on attempt 1 and passed on a later attempt.
- 3If no retry-masked flake is found, the run exits quietly.
- 4For each flake, an action calls the GitHub API to add a `flaky-quarantine` label and comment on the originating PR.
- 5A final action creates a Linear issue titled with the test name, owning team, and a link back to the failed run, tagged `flaky-test`.
Set it up
What you configure once, before turning it on.
- 1Connect GitHubRepos, issues, pull requests, actions.
- 2Connect LinearIssues, projects, cycles, triage.
- 3Set each agent's modelWe leave models unset so you pick the tier — fast + cheap, or top-quality.
- 4Tune it to your dataEdit the prompts, filters, and field mappings so it matches how your team works.
- 5Test, then turn it onRun once against a sample, confirm the output, then enable the trigger.
More Engineering workflows
Gate breaking API PRs behind downstream consumer acknowledgement
When a PR introduces a breaking contract change, comments the impact summary back on the PR, applies a blocking label.
Publish a versioned API changelog to Confluence on each release tag
On a new semver release tag, gathers the contract changes since the last release and writes a clean.
Agent reviews model-license fit and suggests compliant swaps on the PR
When a PR adds a Hugging Face model, an agent reads the model card and license, judges fit against your commercial-use policy.
Upgrade Impact Router to Module Code Owners
Maps a dependency-bump PR's affected modules to their CODEOWNERS, then DMs each owner on Slack with only the changelog slice that touches code they own.
Re-Voice IVR Prompts on Phone-Tree Config Merge
When a phone-tree config change merges in GitHub, regenerates the ElevenLabs audio for any prompt whose script changed in the diff and opens a follow-up PR adding the new audio…
Upstream Release to Notion Upgrade Brief
When a watched package publishes a new release, fetches the release notes, maps them to the internal modules that depend on it.
Run it inside a business
This workflow drops into a full company template. Import the org, and this is one of the playbooks its agents run.
Run this workflow in your colony.
14-day trial. No DevOps. No Sales call. Provisioned in under a minute.