SUMMARIZATION

Severe-Regression Detector with PagerDuty Escalation

On each deploy, summarizes newly introduced Sentry clusters and routes them by severity: high-impact regressions page on-call via PagerDuty, the rest go to a Slack digest.

CategorySummarization
Enginesim
Difficultyadvanced
Triggerwebhook
Steps6
Setup~25 min

How it runs

The automated pipeline, trigger to output.

  • TriggerVercel deploy webhook with release versionVercelVercel
  • ActionFetch Sentry issues first-seen in this releaseSentrySentry
  • ActionScore and summarize each cluster's severityOpenAI
  • LogicBranch on severity against the page threshold
  • ActionPage on-call for high-impact regressionsPagerDutyPagerDuty
  • OutputSend the remaining clusters to a Slack digestSlack

What it does

This workflow grades the error clusters a release introduces and reacts differently to each tier. A regression spiking across many users or hitting a critical path gets an immediate PagerDuty page with a summarized incident description. Lower-severity new clusters are batched into a Slack digest so on-call isn't woken for noise. The decision logic, not a human, decides what is worth a page.

When to use it

Use it when some deploy regressions are genuine incidents and others are minor, and you need automatic separation. Best for teams with on-call rotations who want paging reserved for real impact while still tracking the long tail.

How it works

A Vercel deploy webhook provides the release. The flow fetches Sentry issues first-seen in that release and the model scores each cluster on event rate, user count, and whether it hits a flagged critical surface, producing a severity label and a one-line summary. A branch splits the clusters: anything above the page threshold triggers a PagerDuty incident with the summary and Sentry link attached; everything below is collected into a Slack digest so the full set still gets visibility.

Set it up

What you configure once, before turning it on.

  1. 1
    Connect VercelDeploys, runtime logs, analytics.
  2. 2
    Connect SentryErrors, performance, releases.
  3. 3
    Connect OpenAIModels, embeddings, files.
  4. 4
    Connect PagerDutyIncidents, on-call, escalations.
  5. 5
    Connect SlackChannels, DMs, threads, mentions.
  6. 6
    Set each agent's modelWe leave models unset so you pick the tier — fast + cheap, or top-quality.
  7. 7
    Tune it to your dataEdit the prompts, filters, and field mappings so it matches how your team works.
  8. 8
    Test, then turn it onRun once against a sample, confirm the output, then enable the trigger.

Run this workflow in your colony.

14-day trial. No DevOps. No Sales call. Provisioned in under a minute.