SUMMARIZATION

Route brand-new error classes by severity after release

Identifies error classes that appear for the first time in a release, has an LLM judge their severity from the stack trace and event volume.

CategorySummarization
Enginesim
Difficultyadvanced
Triggerwebhook
Steps6
Setup~25 min

How it runs

The automated pipeline, trigger to output.

  • TriggerVercel deployment succeeded webhookVercelVercel
  • ActionFetch first-seen Sentry issues in the windowSentrySentry
  • ActionSummarize and severity-score each new classOpenAI
  • LogicBranch on assigned severity
  • OutputPage on-call for high severityPagerDutyPagerDuty
  • OutputFile backlog ticket for the restLinearLinear

What it does

This workflow isolates the truly new errors a release introduced (issues with no history before the deploy), then triages them. An LLM reads each new issue's message, stack trace, and event rate and assigns a severity. High-severity issues escalate to PagerDuty; everything else becomes a Linear backlog item. The decision is explained in plain English so the routing is auditable.

When to use it

Use it when a deploy can introduce both a five-alarm fire and harmless new log noise, and you want the fire to page someone while the noise just gets queued. It keeps on-call alerts meaningful without ignoring the long tail.

How it works

A Vercel deploy-succeeded webhook supplies the release. The flow fetches Sentry issues first-seen inside the window. An OpenAI step summarizes and severity-scores each new class. A branch splits on the assigned severity: high-severity issues trigger a PagerDuty incident with the summary as context, while medium and low issues are written to Linear as backlog tickets.

Set it up

What you configure once, before turning it on.

  1. 1
    Connect VercelDeploys, runtime logs, analytics.
  2. 2
    Connect SentryErrors, performance, releases.
  3. 3
    Connect OpenAIModels, embeddings, files.
  4. 4
    Connect PagerDutyIncidents, on-call, escalations.
  5. 5
    Connect LinearIssues, projects, cycles, triage.
  6. 6
    Set each agent's modelWe leave models unset so you pick the tier — fast + cheap, or top-quality.
  7. 7
    Tune it to your dataEdit the prompts, filters, and field mappings so it matches how your team works.
  8. 8
    Test, then turn it onRun once against a sample, confirm the output, then enable the trigger.

Run this workflow in your colony.

14-day trial. No DevOps. No Sales call. Provisioned in under a minute.