ENGINEERING
Auto-Quarantine Flaky Test via PR with Slack Review
When a test is confirmed flaky three times, an agent opens a GitHub pull request that marks the test as skipped with a tracking comment.
How it runs
The automated pipeline, trigger to output.
- TriggerFlaky-count threshold webhookHTTP webhook
- ActionLocate test definition in repoGitHub
- ActionOpen quarantine PR with skip annotationGitHub
- ActionLink tracking ClickUp item in PR bodyClickUp
- LogicAwait Slack approve or reject
- OutputLabel PR ready or close branchGitHub
What it does
It automatically takes a repeatedly flaky test out of the blocking path. After a test trips the flaky signature a configured number of times, an agent branches the repo, annotates the test with a skip marker and a tracking reference, opens a pull request, and asks engineers in Slack to review the quarantine before merge.
When to use it
Use it when flaky tests keep blocking otherwise-green PRs and you want a fast, auditable way to sideline them without a human writing the skip by hand. Humans still approve, so nothing leaves the suite silently.
How it works
- 1A webhook fires when a flaky test crosses the configured failure count.
- 2The agent locates the test definition in the GitHub repo.
- 3It creates a branch and edits the test to add a skip annotation plus a link to the tracking ClickUp item.
- 4It opens a pull request describing the flakiness evidence.
- 5It posts the PR to Slack with approve and reject actions.
- 6On approval the PR is labeled ready to merge; rejection closes the branch.
Set it up
What you configure once, before turning it on.
- 1Connect GitHubRepos, issues, pull requests, actions.
- 2Connect SlackChannels, DMs, threads, mentions.
- 3Connect ClickUpDocs + tasks + chats in one workspace.
- 4Connect HTTP webhookTrigger any URL on agent actions.
- 5Set each agent's modelWe leave models unset so you pick the tier — fast + cheap, or top-quality.
- 6Tune it to your dataEdit the prompts, filters, and field mappings so it matches how your team works.
- 7Test, then turn it onRun once against a sample, confirm the output, then enable the trigger.
More Engineering workflows
Agent reviews model-license fit and suggests compliant swaps on the PR
When a PR adds a Hugging Face model, an agent reads the model card and license, judges fit against your commercial-use policy.
Block PRs that add incompatible Hugging Face model licenses
When a pull request adds or bumps a Hugging Face model dependency, it fetches the model card license, checks it against your org's allowed-license policy.
Quarterly Logging Hygiene Audit Agent
An agent-driven quarterly sweep that surveys all Axiom datasets, builds a logging-hygiene scorecard per service.
Post-Merge Log Volume Recheck After Downsampling PR
After a log-level PR merges, waits a day then re-queries Axiom to confirm the targeted stream's volume actually dropped.
Axiom Ingest Cost Spike to Linear Triage Ticket
When Axiom ingest volume spikes beyond its baseline, identifies which service caused it and files a Linear ticket with the offending log stream, sample lines, and a downsampling…
File a Linear license-review ticket for risky model adds
When a PR introduces a Hugging Face model with a non-permissive or unknown license, it opens a Linear issue assigned to the legal-review team with the model, license.
Run it inside a business
This workflow drops into a full company template. Import the org, and this is one of the playbooks its agents run.
Run this workflow in your colony.
14-day trial. No DevOps. No Sales call. Provisioned in under a minute.