ENGINEERING
Human-in-the-Loop Flake Quarantine: Slack Approval Before Skipping
On a repeated CI failure, posts a Slack message with retry-or-skip buttons and the LLM's recommendation.
How it runs
The automated pipeline, trigger to output.
- TriggerGitHub webhook: test fails second time in a rowGitHub
- ActionSummarize failure and recommend retry vs. skipOpenAI
- ActionPost Slack message with Retry/Skip buttonsSlack
- LogicBranch on engineer's button choice
- ActionRe-dispatch CI job (Retry) or open ticket (Skip)GitHub
- OutputReply in Slack thread with ticket and MR linksSlack
What it does
This workflow keeps a human in the loop. When a test fails repeatedly, it summarizes the failure, asks an LLM for a retry-or-skip recommendation, and posts both to Slack with action buttons. Nothing is quarantined until an engineer clicks, so risky skips never happen silently.
When to use it
Use it on teams that want automation to do the gathering and reasoning but reserve the quarantine decision for a person. Ideal for critical repos where skipping the wrong test could mask a real failure.
How it works
- 1A GitHub webhook fires when a job fails for the second consecutive time on the same test.
- 2An OpenAI step writes a short failure summary plus a recommendation (retry transient infra vs. skip persistent flake).
- 3A Slack message posts the summary, recommendation, and Retry / Skip buttons to the team channel.
- 4A logic branch reads the button clicked: Retry re-dispatches the GitHub workflow; Skip proceeds.
- 5On Skip, it opens a Linear flake ticket and a draft skip MR, then replies in-thread with the links.
Set it up
What you configure once, before turning it on.
- 1Connect GitHubRepos, issues, pull requests, actions.
- 2Connect OpenAIModels, embeddings, files.
- 3Connect SlackChannels, DMs, threads, mentions.
- 4Connect LinearIssues, projects, cycles, triage.
- 5Set each agent's modelWe leave models unset so you pick the tier — fast + cheap, or top-quality.
- 6Tune it to your dataEdit the prompts, filters, and field mappings so it matches how your team works.
- 7Test, then turn it onRun once against a sample, confirm the output, then enable the trigger.
More Engineering workflows
Agent reviews model-license fit and suggests compliant swaps on the PR
When a PR adds a Hugging Face model, an agent reads the model card and license, judges fit against your commercial-use policy.
Block PRs that add incompatible Hugging Face model licenses
When a pull request adds or bumps a Hugging Face model dependency, it fetches the model card license, checks it against your org's allowed-license policy.
Quarterly Logging Hygiene Audit Agent
An agent-driven quarterly sweep that surveys all Axiom datasets, builds a logging-hygiene scorecard per service.
Post-Merge Log Volume Recheck After Downsampling PR
After a log-level PR merges, waits a day then re-queries Axiom to confirm the targeted stream's volume actually dropped.
Axiom Ingest Cost Spike to Linear Triage Ticket
When Axiom ingest volume spikes beyond its baseline, identifies which service caused it and files a Linear ticket with the offending log stream, sample lines, and a downsampling…
File a Linear license-review ticket for risky model adds
When a PR introduces a Hugging Face model with a non-permissive or unknown license, it opens a Linear issue assigned to the legal-review team with the model, license.
Run it inside a business
This workflow drops into a full company template. Import the org, and this is one of the playbooks its agents run.
Run this workflow in your colony.
14-day trial. No DevOps. No Sales call. Provisioned in under a minute.