AI AGENTS
On-Call Agent: Slack Slash-Command On-Demand Service Diagnosis
An engineer types a slash command naming a service in Slack, and the agent pulls its live metrics and recent changes, returns a diagnosis with proposed next steps in the thread.
How it runs
The automated pipeline, trigger to output.
- TriggerSlash command invoked in SlackSlack
- ActionQuery current metrics from DatadogDatadog
- ActionPull recent changes from GitHubGitHub
- LogicBuild diagnosis and rank remediation options
- OutputReply in Slack thread with optionsSlack
What it does
Gives every engineer an on-demand diagnostician inside Slack. Type the command with a service name and the agent investigates on the spot, summarizing what looks wrong and what it would try, without leaving chat.
When to use it
Use it when someone notices something off but no monitor has fired yet, or when you want a quick second opinion during an active incident. It's the fastest path from a hunch to a structured investigation.
How it works
- 1An engineer invokes the slash command in Slack with the target service name.
- 2The agent queries Datadog for that service's current error, latency, and resource metrics.
- 3It pulls the latest commits and deploys for the service from GitHub to see what changed recently.
- 4Logic assembles a diagnosis and a short menu of proposed remediation options ranked by likely impact.
- 5The agent replies in the same Slack thread with the findings and the options, leaving the decision and execution to the requesting engineer.
Set it up
What you configure once, before turning it on.
- 1Connect SlackChannels, DMs, threads, mentions.
- 2Connect DatadogMetrics, traces, log search.
- 3Connect GitHubRepos, issues, pull requests, actions.
- 4Set each agent's modelWe leave models unset so you pick the tier — fast + cheap, or top-quality.
- 5Tune it to your dataEdit the prompts, filters, and field mappings so it matches how your team works.
- 6Test, then turn it onRun once against a sample, confirm the output, then enable the trigger.
More AI Agents workflows
Custom Metrics Cardinality Spike Pager
A webhook from a Datadog monitor fires when custom-metric cardinality jumps; an agent pinpoints the offending metric and tag, estimates the added cost.
Sentry-to-Confluence Runbook Updater
When a Sentry issue is resolved, the agent finds the matching Confluence runbook page and proposes an inline update with the verified fix.
Stale Doc-PR Chaser for Runbook Gaps
On a daily schedule the agent finds runbook doc PRs that were opened from resolved incidents but never reviewed, summarizes what each one fixes.
Resolved Incident to Public Troubleshooting Doc
For customer-facing errors resolved in Sentry, the agent drafts a sanitized troubleshooting entry and opens a PR to your ReadMe documentation.
On-Call Runbook Gap Closer: Resolved Sentry Issues to Doc PRs
An agent reads each newly resolved Sentry issue, compares the actual fix against your existing runbook, and opens a GitHub PR adding the missing remediation steps.
Weekly On-Call Doc-Gap Digest
Each week the agent reviews every Sentry issue resolved in the last 7 days, ranks the ones whose runbook coverage is missing or thin.
Run it inside a business
This workflow drops into a full company template. Import the org, and this is one of the playbooks its agents run.
Run this workflow in your colony.
14-day trial. No DevOps. No Sales call. Provisioned in under a minute.