AI AGENTS
Datadog Custom Metrics Budget Watchdog
Hourly agent that tracks Datadog custom metric counts against your contracted budget and alerts Slack with the top tag dimensions driving the overage before billing surprises hit.
How it runs
The automated pipeline, trigger to output.
- TriggerHourly schedule fires
- ActionPull Datadog custom metric counts and tag cardinalityDatadog
- LogicCompare against budget and burn rate
- LogicIdentify top metric and tag combinations driving overage
- OutputSend Slack alert with offending dimensions and suggested fixSlack
What it does
Datadog bills on custom metrics, and a single high-cardinality tag can quietly multiply your billable timeseries overnight. This agent polls your custom metric usage hourly, compares it against the budget you set, and when you cross a threshold it surfaces exactly which metric names and tag keys are responsible so you can act before the monthly true-up.
When to use it
Use it if you've ever been surprised by a Datadog custom metrics overage bill. Ideal for teams on a committed custom-metrics tier who want an early-warning trip wire rather than a postmortem.
How it works
- 1An hourly schedule triggers the check.
- 2The agent pulls active custom metric counts and per-metric tag cardinality from the Datadog API.
- 3A logic step compares the current billable timeseries total against your configured budget and burn rate.
- 4If usage is on track to exceed budget, it identifies the top metric-and-tag combinations contributing to the spike.
- 5It sends a Slack alert listing the offending dimensions, current versus budgeted count, and a suggested tag to drop or aggregate.
Set it up
What you configure once, before turning it on.
- 1Connect DatadogMetrics, traces, log search.
- 2Connect SlackChannels, DMs, threads, mentions.
- 3Set each agent's modelWe leave models unset so you pick the tier — fast + cheap, or top-quality.
- 4Tune it to your dataEdit the prompts, filters, and field mappings so it matches how your team works.
- 5Test, then turn it onRun once against a sample, confirm the output, then enable the trigger.
More AI Agents workflows
Custom Metrics Cardinality Spike Pager
A webhook from a Datadog monitor fires when custom-metric cardinality jumps; an agent pinpoints the offending metric and tag, estimates the added cost.
Sentry-to-Confluence Runbook Updater
When a Sentry issue is resolved, the agent finds the matching Confluence runbook page and proposes an inline update with the verified fix.
Stale Doc-PR Chaser for Runbook Gaps
On a daily schedule the agent finds runbook doc PRs that were opened from resolved incidents but never reviewed, summarizes what each one fixes.
Resolved Incident to Public Troubleshooting Doc
For customer-facing errors resolved in Sentry, the agent drafts a sanitized troubleshooting entry and opens a PR to your ReadMe documentation.
On-Call Runbook Gap Closer: Resolved Sentry Issues to Doc PRs
An agent reads each newly resolved Sentry issue, compares the actual fix against your existing runbook, and opens a GitHub PR adding the missing remediation steps.
Weekly On-Call Doc-Gap Digest
Each week the agent reviews every Sentry issue resolved in the last 7 days, ranks the ones whose runbook coverage is missing or thin.
Run it inside a business
This workflow drops into a full company template. Import the org, and this is one of the playbooks its agents run.
Run this workflow in your colony.
14-day trial. No DevOps. No Sales call. Provisioned in under a minute.