ENGINEERING
Auto-Close Quarantine Issues When a Test Goes Green Again
Runs nightly, checks each open quarantine issue against recent Datadog results, and closes the Linear issue plus removes the GitHub quarantine label once the test has passed…
How it runs
The automated pipeline, trigger to output.
- TriggerNightly schedule fires
- ActionList open Linear quarantine issues and test idsLinear
- ActionQuery Datadog for recent pass/fail history per testDatadog
- LogicKeep tests with N consecutive clean passes
- ActionRemove flaky-quarantine label on GitHubGitHub
- OutputClose matching Linear issues with green-streak noteLinear
What it does
Keeps the quarantine list honest. Many flaky-test issues get fixed but the ticket and the skip linger forever. This sweeps open quarantine issues, verifies the test has been green for a configurable streak, and automatically retires the quarantine — closing the issue and removing the label.
When to use it
Use it when your quarantine backlog never shrinks and stale skips are eroding real coverage. It garbage-collects resolved flakes without manual bookkeeping.
How it works
- 1A nightly schedule triggers the sweep.
- 2An action lists open Linear issues tagged `flaky-test` along with their test identifiers.
- 3An action queries Datadog for each test's recent pass/fail history.
- 4A logic step keeps only tests with N consecutive clean passes and no recent failures.
- 5An action removes the `flaky-quarantine` label from GitHub for those tests.
- 6An output closes the matching Linear issues with a comment noting the green streak.
Set it up
What you configure once, before turning it on.
- 1Connect LinearIssues, projects, cycles, triage.
- 2Connect DatadogMetrics, traces, log search.
- 3Connect GitHubRepos, issues, pull requests, actions.
- 4Set each agent's modelWe leave models unset so you pick the tier — fast + cheap, or top-quality.
- 5Tune it to your dataEdit the prompts, filters, and field mappings so it matches how your team works.
- 6Test, then turn it onRun once against a sample, confirm the output, then enable the trigger.
More Engineering workflows
Gate breaking API PRs behind downstream consumer acknowledgement
When a PR introduces a breaking contract change, comments the impact summary back on the PR, applies a blocking label.
Publish a versioned API changelog to Confluence on each release tag
On a new semver release tag, gathers the contract changes since the last release and writes a clean.
Agent reviews model-license fit and suggests compliant swaps on the PR
When a PR adds a Hugging Face model, an agent reads the model card and license, judges fit against your commercial-use policy.
Upgrade Impact Router to Module Code Owners
Maps a dependency-bump PR's affected modules to their CODEOWNERS, then DMs each owner on Slack with only the changelog slice that touches code they own.
Re-Voice IVR Prompts on Phone-Tree Config Merge
When a phone-tree config change merges in GitHub, regenerates the ElevenLabs audio for any prompt whose script changed in the diff and opens a follow-up PR adding the new audio…
Upstream Release to Notion Upgrade Brief
When a watched package publishes a new release, fetches the release notes, maps them to the internal modules that depend on it.
Run it inside a business
This workflow drops into a full company template. Import the org, and this is one of the playbooks its agents run.
Run this workflow in your colony.
14-day trial. No DevOps. No Sales call. Provisioned in under a minute.