DEVOPS
Agent triages flaky test logs and proposes a fix
When a test is quarantined, an agent reads its recent failure logs, infers the likely root cause (timing, ordering, network, fixture), drafts a remediation plan.
How it runs
The automated pipeline, trigger to output.
- TriggerTest labeled quarantineGitHub
- ActionFetch failing logs and stack tracesGitHub
- ActionPull related traces from DatadogDatadog
- LogicAgent infers root cause and drafts fixOpenAI
- OutputPost remediation plan to Linear ticketLinear
What it does
This template puts an investigation agent on every newly quarantined test. It gathers the test's recent failure logs and stack traces, reasons about the most likely flakiness category such as a race condition, test-order dependency, network timeout, or shared fixture, and drafts a concrete remediation plan attached to the tracking ticket.
When to use it
Use it when quarantine tickets sit empty because nobody has time to dig into intermittent logs. The agent does the first-pass diagnosis so the assigned engineer starts with a hypothesis instead of a blank page.
How it works
- 1A GitHub label event for `quarantine` on an issue fires the trigger.
- 2The agent fetches recent failing-run logs and stack traces for the named test via GitHub.
- 3It pulls additional context such as related test traces from Datadog where available.
- 4The agent reasons over the evidence to classify the flake type and draft a fix plan with confidence and next steps.
- 5The plan is posted as a comment on the Linear tracking ticket for the owner to act on.
Set it up
What you configure once, before turning it on.
- 1Connect GitHubRepos, issues, pull requests, actions.
- 2Connect DatadogMetrics, traces, log search.
- 3Connect LinearIssues, projects, cycles, triage.
- 4Connect OpenAIModels, embeddings, files.
- 5Set each agent's modelWe leave models unset so you pick the tier — fast + cheap, or top-quality.
- 6Tune it to your dataEdit the prompts, filters, and field mappings so it matches how your team works.
- 7Test, then turn it onRun once against a sample, confirm the output, then enable the trigger.
More DevOps workflows
Hugging Face Spaces idle-runtime sweep with auto-pause
On a schedule, scans all Hugging Face Spaces for ones running idle past a threshold, pauses them to stop billing, and posts a Slack summary with the estimated monthly savings.
Slack-approved pause for idle Hugging Face Spaces
On a daily scan it finds idle paid Spaces and posts an interactive Slack approval; on approve it pauses the Space and logs the decision to a GitHub issue audit trail.
Generate a weekly de-flake report and assign Linear cleanup tickets
On a weekly schedule, aggregates the current quarantine manifest and recent flake history, builds a prioritized report.
Block costly Hugging Face Space hardware upgrades in PR review
When a pull request changes a Space's hardware config, it estimates the new monthly cost and posts a GitHub PR comment that flags upgrades crossing a budget ceiling.
Auto-release tests from quarantine once they prove stable
Triggered by a webhook from a nightly stability runner, checks whether quarantined tests have passed enough consecutive runs, removes the stable ones from quarantine in GitHub.
Quarantine a test on demand from a PR comment command
Triggered when an engineer comments a quarantine command on a pull request, validates the test name, commits the quarantine change to that PR branch, opens a tracking issue.
Run it inside a business
This workflow drops into a full company template. Import the org, and this is one of the playbooks its agents run.
Run this workflow in your colony.
14-day trial. No DevOps. No Sales call. Provisioned in under a minute.