ENGINEERING
Escalate Suite Instability to PagerDuty When Flakes Block Releases
On a schedule, evaluates aggregate flake load from Datadog and the open deflake backlog in Linear, and if the main pipeline's flake rate threatens releases.
How it runs
The automated pipeline, trigger to output.
- TriggerScheduled stability evaluation
- ActionFetch main pipeline flake metrics from DatadogDatadog
- ActionPull open deflake tickets and ages from LinearLinear
- LogicCompute stability score against release-risk threshold
- ActionOpen PagerDuty incident when threshold breachedPagerDuty
- OutputPost incident summary to SlackSlack
What it does
It treats systemic flakiness as an operational risk, not just a backlog item. On a schedule it pulls the main branch's recent CI flake rate from Datadog and the count of aging open deflake tickets from Linear, computes a stability score, and if the suite is unstable enough to jeopardize merges and releases it raises a PagerDuty incident so an owner is accountable in real time.
When to use it
Use it when flakes have crossed from annoyance to release blocker and you need a clear escalation trigger instead of ad-hoc complaints. It draws the line at which test instability becomes an on-call concern.
How it works
- 1A schedule triggers the evaluation.
- 2A Datadog action fetches the main pipeline's recent flake and failure-retry metrics.
- 3A Linear action pulls open deflake tickets and their ages.
- 4A logic step computes a stability score and checks it against the release-risk threshold.
- 5A PagerDuty action opens an incident for the test-platform on-call when the threshold is breached.
- 6A Slack message posts the incident summary and the top contributing flaky tests.
Set it up
What you configure once, before turning it on.
- 1Connect DatadogMetrics, traces, log search.
- 2Connect LinearIssues, projects, cycles, triage.
- 3Connect PagerDutyIncidents, on-call, escalations.
- 4Connect SlackChannels, DMs, threads, mentions.
- 5Set each agent's modelWe leave models unset so you pick the tier — fast + cheap, or top-quality.
- 6Tune it to your dataEdit the prompts, filters, and field mappings so it matches how your team works.
- 7Test, then turn it onRun once against a sample, confirm the output, then enable the trigger.
More Engineering workflows
Agent reviews model-license fit and suggests compliant swaps on the PR
When a PR adds a Hugging Face model, an agent reads the model card and license, judges fit against your commercial-use policy.
Block PRs that add incompatible Hugging Face model licenses
When a pull request adds or bumps a Hugging Face model dependency, it fetches the model card license, checks it against your org's allowed-license policy.
Quarterly Logging Hygiene Audit Agent
An agent-driven quarterly sweep that surveys all Axiom datasets, builds a logging-hygiene scorecard per service.
Post-Merge Log Volume Recheck After Downsampling PR
After a log-level PR merges, waits a day then re-queries Axiom to confirm the targeted stream's volume actually dropped.
Axiom Ingest Cost Spike to Linear Triage Ticket
When Axiom ingest volume spikes beyond its baseline, identifies which service caused it and files a Linear ticket with the offending log stream, sample lines, and a downsampling…
File a Linear license-review ticket for risky model adds
When a PR introduces a Hugging Face model with a non-permissive or unknown license, it opens a Linear issue assigned to the legal-review team with the model, license.
Run it inside a business
This workflow drops into a full company template. Import the org, and this is one of the playbooks its agents run.
Run this workflow in your colony.
14-day trial. No DevOps. No Sales call. Provisioned in under a minute.