ENGINEERING

Triage Honeycomb p99 latency outliers into a Linear perf ticket

On a schedule, queries Honeycomb for the slowest spans in a service, identifies the offending database query.

CategoryEngineering
Enginesim
Difficultyintermediate
Triggerschedule
Steps6
Setup~15 min

How it runs

The automated pipeline, trigger to output.

  • TriggerHourly schedule fires
  • ActionQuery Honeycomb for spans above p99 threshold, grouped by queryHoneycomb
  • LogicPick top offender; skip if under SLO
  • ActionSearch Linear for an existing ticket with this query fingerprintLinearLinear
  • ActionCreate Linear perf ticket with query, metrics, and trace linkLinearLinear
  • OutputPost ticket link to Slack perf channelSlack

What it does

Every hour it pulls the worst-offending spans from a Honeycomb dataset, isolates the specific operation and query driving p99 latency, and opens a Linear performance ticket scoped to that query — only if one does not already exist.

When to use it

When a service has intermittent latency spikes and you want them turned into actionable, owner-assigned tickets instead of dashboards nobody watches. Best for teams who triage perf in Linear and want the slow query and a trace exemplar in the ticket body.

How it works

  1. 1A schedule fires hourly.
  2. 2Query Honeycomb for spans in the target dataset where duration exceeds the p99 threshold, grouped by `db.statement` and `name`.
  3. 3A logic step filters to the top offender and skips runs where the slowest span is under the SLO.
  4. 4Search Linear for an open ticket matching the query fingerprint to avoid duplicates.
  5. 5If none exists, create a Linear issue with the query text, p99/p50 numbers, span count, and a direct Honeycomb trace link.
  6. 6Post the new ticket link to the team's Slack perf channel.

Set it up

What you configure once, before turning it on.

  1. 1
    Connect HoneycombDistributed traces and queries.
  2. 2
    Connect LinearIssues, projects, cycles, triage.
  3. 3
    Connect SlackChannels, DMs, threads, mentions.
  4. 4
    Set each agent's modelWe leave models unset so you pick the tier — fast + cheap, or top-quality.
  5. 5
    Tune it to your dataEdit the prompts, filters, and field mappings so it matches how your team works.
  6. 6
    Test, then turn it onRun once against a sample, confirm the output, then enable the trigger.

Run this workflow in your colony.

14-day trial. No DevOps. No Sales call. Provisioned in under a minute.