ENGINEERING
Fast-Burn Spike Escalation to PagerDuty
On a Datadog fast-burn alert webhook, this workflow confirms the SLO is burning faster than the configured multiplier and pages the on-call engineer through PagerDuty…
How it runs
The automated pipeline, trigger to output.
- TriggerDatadog fast-burn webhookHTTP webhook
- ActionRe-fetch authoritative SLO stateDatadog
- LogicConfirm multiplier and compute hours left
- ActionTrigger PagerDuty incidentPagerDuty
- OutputPost incident link to SlackSlack
What it does
It catches the dangerous case the hourly forecast can miss: a sudden fast burn that will drain the budget in hours, not weeks. It validates the spike against the current SLO state and escalates straight to a human via PagerDuty with the time-to-exhaustion already calculated.
When to use it
Use it for tier-1 services where a fast-burn event needs a page now, not a Slack message someone reads in the morning. Pair it with the hourly forecaster: that one watches slow leaks, this one watches blowouts.
How it works
- 1A Datadog fast-burn monitor fires an HTTP webhook into the workflow.
- 2It re-queries the Datadog SLO to get authoritative remaining budget and current burn rate, avoiding action on a stale alert payload.
- 3A logic step computes hours-to-exhaustion and confirms the burn multiplier exceeds the escalation threshold.
- 4If confirmed, it triggers a PagerDuty incident with severity scaled to how little time remains.
- 5It posts the incident link to the team Slack channel for visibility.
Set it up
What you configure once, before turning it on.
- 1Connect HTTP webhookTrigger any URL on agent actions.
- 2Connect DatadogMetrics, traces, log search.
- 3Connect PagerDutyIncidents, on-call, escalations.
- 4Connect SlackChannels, DMs, threads, mentions.
- 5Set each agent's modelWe leave models unset so you pick the tier — fast + cheap, or top-quality.
- 6Tune it to your dataEdit the prompts, filters, and field mappings so it matches how your team works.
- 7Test, then turn it onRun once against a sample, confirm the output, then enable the trigger.
More Engineering workflows
Agent reviews model-license fit and suggests compliant swaps on the PR
When a PR adds a Hugging Face model, an agent reads the model card and license, judges fit against your commercial-use policy.
Block PRs that add incompatible Hugging Face model licenses
When a pull request adds or bumps a Hugging Face model dependency, it fetches the model card license, checks it against your org's allowed-license policy.
Quarterly Logging Hygiene Audit Agent
An agent-driven quarterly sweep that surveys all Axiom datasets, builds a logging-hygiene scorecard per service.
Post-Merge Log Volume Recheck After Downsampling PR
After a log-level PR merges, waits a day then re-queries Axiom to confirm the targeted stream's volume actually dropped.
Axiom Ingest Cost Spike to Linear Triage Ticket
When Axiom ingest volume spikes beyond its baseline, identifies which service caused it and files a Linear ticket with the offending log stream, sample lines, and a downsampling…
File a Linear license-review ticket for risky model adds
When a PR introduces a Hugging Face model with a non-permissive or unknown license, it opens a Linear issue assigned to the legal-review team with the model, license.
Run it inside a business
This workflow drops into a full company template. Import the org, and this is one of the playbooks its agents run.
Run this workflow in your colony.
14-day trial. No DevOps. No Sales call. Provisioned in under a minute.