ENGINEERING

Datadog error-rate anomaly to PagerDuty + GitHub issue

When Datadog detects an anomalous error-rate climb on a service, it pages the on-call via PagerDuty and opens a GitHub issue pre-filled with the affected service, metric…

CategoryEngineering
Enginesim
Difficultyintermediate
Triggerevent
Steps5
Setup~15 min

How it runs

The automated pipeline, trigger to output.

  • TriggerDatadog monitor enters alert on error-rate anomalyDatadogDatadog
  • LogicFilter for triggering transition, ignore recovery/warn
  • ActionTrigger PagerDuty incident on the service escalation policyPagerDutyPagerDuty
  • ActionOpen GitHub issue with metric snapshot and runbook linkGitHubGitHub
  • OutputAttach GitHub issue URL to the PagerDuty incidentPagerDutyPagerDuty

What it does

Bridges Datadog's anomaly detection to both human escalation and a durable engineering record: the on-call gets paged immediately, and a GitHub issue is opened so the incident has a place to live after the page is acknowledged.

When to use it

Use it for production services where error-rate anomalies need a real human now and a follow-up paper trail. Ideal when GitHub Issues is your source of truth for postmortems and PagerDuty owns escalation.

How it works

  1. 1A Datadog monitor in alert state hits the workflow webhook with the service, metric, and anomaly window.
  2. 2A filter confirms the alert is a triggering transition (not a recovery or warn) before escalating.
  3. 3PagerDuty receives a triggered incident routed to the service's escalation policy.
  4. 4A GitHub issue is created in the service repo with the metric snapshot, time window, dashboard link, and the matching runbook.
  5. 5The PagerDuty incident is updated with the GitHub issue URL so responders jump straight to the tracking record.

Set it up

What you configure once, before turning it on.

  1. 1
    Connect DatadogMetrics, traces, log search.
  2. 2
    Connect PagerDutyIncidents, on-call, escalations.
  3. 3
    Connect GitHubRepos, issues, pull requests, actions.
  4. 4
    Set each agent's modelWe leave models unset so you pick the tier — fast + cheap, or top-quality.
  5. 5
    Tune it to your dataEdit the prompts, filters, and field mappings so it matches how your team works.
  6. 6
    Test, then turn it onRun once against a sample, confirm the output, then enable the trigger.

Run this workflow in your colony.

14-day trial. No DevOps. No Sales call. Provisioned in under a minute.