ENGINEERING
AI flaky-test triage agent: classify root cause and route to the right owner
On a confirmed flaky test, an agent reads the failure logs and diff to classify the root cause (timing, network, ordering, data).
How it runs
The automated pipeline, trigger to output.
- TriggerTest confirmed flakyGitHub
- ActionGather logs, test source, recent commitsGitHub
- LogicAgent classifies root cause and drafts fixOpenAI
- ActionSelect owner from CODEOWNERS + authorshipGitHub
- OutputFile categorized, owner-assigned Linear ticketLinear
What it does
Adds reasoning on top of detection. Once a test is confirmed flaky, an agent inspects the stack trace, recent diffs, and rerun history to guess why it flakes (race condition, network timeout, test-order dependency, shared fixture) and writes a triage note with a suggested fix and the best owner to assign.
When to use it
Use when raw flaky tickets pile up unread because nobody knows the cause. The agent does the first pass of investigation a senior engineer would, so the ticket arrives with a hypothesis and a named owner instead of just a red log.
How it works
- 1A GitHub webhook fires when a test is confirmed flaky (pass-on-retry).
- 2The flow gathers context: failure logs, the failing test source, and recent commits touching it.
- 3The agent classifies the likely root cause and drafts a suggested fix and confidence level.
- 4It chooses an owner from CODEOWNERS plus recent authorship signals.
- 5It files a Linear ticket with the category label, root-cause hypothesis, and proposed fix, assigned to that owner.
Set it up
What you configure once, before turning it on.
- 1Connect GitHubRepos, issues, pull requests, actions.
- 2Connect LinearIssues, projects, cycles, triage.
- 3Connect OpenAIModels, embeddings, files.
- 4Set each agent's modelWe leave models unset so you pick the tier — fast + cheap, or top-quality.
- 5Tune it to your dataEdit the prompts, filters, and field mappings so it matches how your team works.
- 6Test, then turn it onRun once against a sample, confirm the output, then enable the trigger.
More Engineering workflows
Upgrade Impact Router to Module Code Owners
Maps a dependency-bump PR's affected modules to their CODEOWNERS, then DMs each owner on Slack with only the changelog slice that touches code they own.
Re-Voice IVR Prompts on Phone-Tree Config Merge
When a phone-tree config change merges in GitHub, regenerates the ElevenLabs audio for any prompt whose script changed in the diff and opens a follow-up PR adding the new audio…
Agent reviews model-license fit and suggests compliant swaps on the PR
When a PR adds a Hugging Face model, an agent reads the model card and license, judges fit against your commercial-use policy.
Scan for deprecated endpoints and email consumers a weekly sunset countdown
On a weekly schedule, scans the OpenAPI spec for endpoints marked deprecated with a sunset date, and emails each consuming team a countdown of how many days remain before removal.
Publish a versioned API changelog to Confluence on each release tag
On a new semver release tag, gathers the contract changes since the last release and writes a clean.
Gate breaking API PRs behind downstream consumer acknowledgement
When a PR introduces a breaking contract change, comments the impact summary back on the PR, applies a blocking label.
Run it inside a business
This workflow drops into a full company template. Import the org, and this is one of the playbooks its agents run.
Run this workflow in your colony.
14-day trial. No DevOps. No Sales call. Provisioned in under a minute.