DEVOPS
Auto-quarantine flaky tests from CI failures and file a tracking issue
Watches GitHub Actions test runs for tests that pass on retry but failed on first attempt, marks them as quarantined in the suite.
How it runs
The automated pipeline, trigger to output.
- TriggerGitHub Actions workflow run completesGitHub
- LogicDetect tests that failed first attempt but passed on retry
- LogicSkip tests that already have an open quarantine issue
- ActionCommit quarantine manifest update to isolate the testGitHub
- ActionOpen GitHub tracking issue with failure historyGitHub
- OutputPost quarantine summary to SlackSlack
What it does
It catches tests that fail intermittently in CI, isolates them so they stop blocking merges, and opens a GitHub issue that records the offending test, its failure rate, and the runs where it flaked.
When to use it
Use it when a green build keeps getting blocked by one or two unreliable tests and your team wants those tests pulled out of the gating path automatically instead of someone manually adding `@skip` and forgetting to track it.
How it works
- 1A completed GitHub Actions workflow run fires the trigger with its test report attached.
- 2The flow parses the JUnit results and compares first-attempt outcomes against retry outcomes to find tests that failed then passed.
- 3A logic step checks whether each flaky test already has an open quarantine issue to avoid duplicates.
- 4For new offenders it commits a change to the quarantine manifest file in the repo, moving the test out of the required suite.
- 5It opens a GitHub issue labeled `flaky-test` with the failure history and links the commit.
- 6It posts a short summary to the team channel so engineers know what was isolated.
Set it up
What you configure once, before turning it on.
- 1Connect GitHubRepos, issues, pull requests, actions.
- 2Connect SlackChannels, DMs, threads, mentions.
- 3Set each agent's modelWe leave models unset so you pick the tier — fast + cheap, or top-quality.
- 4Tune it to your dataEdit the prompts, filters, and field mappings so it matches how your team works.
- 5Test, then turn it onRun once against a sample, confirm the output, then enable the trigger.
More DevOps workflows
Block costly Hugging Face Space hardware upgrades in PR review
When a pull request changes a Space's hardware config, it estimates the new monthly cost and posts a GitHub PR comment that flags upgrades crossing a budget ceiling.
Auto-spin a Zoom war-room when PagerDuty hits SEV-1
When a PagerDuty incident escalates to a critical severity, this workflow creates a dedicated Zoom meeting and posts the bridge link to the incident's Slack channel so responders…
Page on-call when a Hugging Face Space build is stuck or errored
Polls Hugging Face Space runtime status on a schedule and opens a PagerDuty incident when a Space sits in a build or error state past a deadline, with a Slack heads-up.
Slack-approved pause for idle Hugging Face Spaces
On a daily scan it finds idle paid Spaces and posts an interactive Slack approval; on approve it pauses the Space and logs the decision to a GitHub issue audit trail.
Hugging Face Spaces idle-runtime sweep with auto-pause
On a schedule, scans all Hugging Face Spaces for ones running idle past a threshold, pauses them to stop billing, and posts a Slack summary with the estimated monthly savings.
Open a Zoom war-room from a Datadog multi-alert storm
When a Datadog monitor crosses a critical threshold, this workflow dedupes against active incidents, and only for a genuinely new outage it creates a Zoom bridge.
Run it inside a business
This workflow drops into a full company template. Import the org, and this is one of the playbooks its agents run.
Run this workflow in your colony.
14-day trial. No DevOps. No Sales call. Provisioned in under a minute.