ENGINEERING

Critical-path slow trace → Sentry correlation → PagerDuty escalation

When a trace on a critical service exceeds a tight latency budget and correlates with a spike in Sentry errors, this workflow opens a PagerDuty incident and attaches the joined…

CategoryEngineering
Enginesim
Difficultyadvanced
Triggerwebhook
Steps5
Setup~25 min

How it runs

The automated pipeline, trigger to output.

  • TriggerHoneycomb slow-trace alert on critical serviceHoneycomb
  • LogicVerify critical-path allowlist + tight budget overage
  • ActionQuery Sentry and gate on error-rate spikeSentrySentry
  • ActionCreate GitLab issue with joined evidenceGitLabGitLab
  • OutputOpen linked PagerDuty incident, route to policyPagerDutyPagerDuty

What it does

Distinguishes a real outage from a slow page. For traces on protected critical-path services, it checks Honeycomb latency against a tight budget, confirms a correlated Sentry error surge, and — only then — escalates to PagerDuty while filing a linked GitLab issue so responders have evidence the moment they're paged.

When to use it

Use for your most important services where slow + erroring means customer impact. Best when you want paging reserved for correlated signals, not raw latency, to cut alert fatigue while protecting SLOs.

How it works

  1. 1A Honeycomb trigger fires on a slow trace tagged to a critical service.
  2. 2The workflow verifies the service is on the critical-path allowlist and the overage clears the tight budget.
  3. 3It queries Sentry and gates on whether error rate spiked above baseline in the window.
  4. 4If both conditions hold, it creates a GitLab issue with the joined trace and error context.
  5. 5It opens a PagerDuty incident referencing the issue and routes to the owning escalation policy.

Set it up

What you configure once, before turning it on.

  1. 1
    Connect HoneycombDistributed traces and queries.
  2. 2
    Connect SentryErrors, performance, releases.
  3. 3
    Connect PagerDutyIncidents, on-call, escalations.
  4. 4
    Connect GitLabRepos, MRs, pipelines, registry.
  5. 5
    Set each agent's modelWe leave models unset so you pick the tier — fast + cheap, or top-quality.
  6. 6
    Tune it to your dataEdit the prompts, filters, and field mappings so it matches how your team works.
  7. 7
    Test, then turn it onRun once against a sample, confirm the output, then enable the trigger.

Run this workflow in your colony.

14-day trial. No DevOps. No Sales call. Provisioned in under a minute.