ENGINEERING

Correlate flaky-test failures with Honeycomb traces before filing

When a quarantine candidate is detected, pulls the matching CI test-run traces from Honeycomb to find shared timeouts, slow spans, or resource contention.

CategoryEngineering
Enginesim
Difficultyadvanced
Triggerwebhook
Steps4
Setup~25 min

How it runs

The automated pipeline, trigger to output.

  • TriggerWebhook: quarantine candidate detectedHTTP webhook
  • ActionQuery Honeycomb traces for failing runsHoneycomb
  • LogicIsolate recurring slow or erroring span
  • OutputFile enriched Linear issue with root-cause signalLinearLinear

What it does

Goes beyond "this test is flaky" to "here is why." For each quarantine candidate it inspects the distributed traces emitted by the test run, surfaces the common slow or erroring spans across failures, and attaches that evidence to the bug so an engineer starts with a hypothesis instead of a blank page.

When to use it

Use this when your test suite emits OpenTelemetry traces to Honeycomb and flakiness is suspected to come from timeouts, external calls, or contention rather than bad assertions. It makes triage dramatically faster.

How it works

  1. 1A webhook fires when the classifier marks a spec as a quarantine candidate, passing the spec name and recent failing run IDs.
  2. 2The flow queries Honeycomb for the traces tied to those run IDs, filtered to the failing test's spans.
  3. 3A logic step compares spans across the failing runs to isolate the recurring slow or error-status span.
  4. 4It composes a root-cause summary from the shared signal.
  5. 5A Linear issue is created with the spec, flake rate, the suspect span, and a Honeycomb query link, routed to the owning team.

Set it up

What you configure once, before turning it on.

  1. 1
    Connect HTTP webhookTrigger any URL on agent actions.
  2. 2
    Connect HoneycombDistributed traces and queries.
  3. 3
    Connect LinearIssues, projects, cycles, triage.
  4. 4
    Set each agent's modelWe leave models unset so you pick the tier — fast + cheap, or top-quality.
  5. 5
    Tune it to your dataEdit the prompts, filters, and field mappings so it matches how your team works.
  6. 6
    Test, then turn it onRun once against a sample, confirm the output, then enable the trigger.

Run this workflow in your colony.

14-day trial. No DevOps. No Sales call. Provisioned in under a minute.