ENGINEERING
Nightly Flaky-Test Rollup from Honeycomb with Owner Triage
Each night queries Honeycomb test-run telemetry for tests whose pass/fail outcome varied across identical inputs, ranks them by flake rate.
How it runs
The automated pipeline, trigger to output.
- TriggerNightly schedule fires
- ActionQuery Honeycomb for test-run events and outcomesHoneycomb
- LogicCompute flake rate and filter above threshold
- ActionUpsert one consolidated Linear ticket per owning teamLinear
- OutputPost ranked flake digest to engineering SlackSlack
What it does
This workflow turns raw test telemetry into an actionable triage list. It queries Honeycomb for test spans over the last 24 hours, identifies tests that both passed and failed under the same conditions, and computes a flake rate per test. Tests above a threshold are grouped by owning team and rolled into one Linear ticket each, updated daily so noise stays consolidated rather than spawning duplicate issues.
When to use it
Use this when your test suite emits run-level telemetry to Honeycomb and you want a daily, data-driven view of which tests are flakiest and who owns them, without a ticket storm.
How it works
- 1A nightly schedule triggers the run.
- 2The flow queries Honeycomb for test-run events grouped by test name and outcome over 24 hours.
- 3A logic step computes flake rate per test and filters to those above the configured threshold.
- 4Each flaky test is mapped to an owning team via a lookup table.
- 5For each team, the flow upserts a single Linear ticket: created if absent, updated with the latest ranked list if it already exists.
- 6A digest of the top flakes is posted to the engineering Slack channel.
Set it up
What you configure once, before turning it on.
- 1Connect HoneycombDistributed traces and queries.
- 2Connect LinearIssues, projects, cycles, triage.
- 3Connect SlackChannels, DMs, threads, mentions.
- 4Set each agent's modelWe leave models unset so you pick the tier — fast + cheap, or top-quality.
- 5Tune it to your dataEdit the prompts, filters, and field mappings so it matches how your team works.
- 6Test, then turn it onRun once against a sample, confirm the output, then enable the trigger.
More Engineering workflows
Gate breaking API PRs behind downstream consumer acknowledgement
When a PR introduces a breaking contract change, comments the impact summary back on the PR, applies a blocking label.
Publish a versioned API changelog to Confluence on each release tag
On a new semver release tag, gathers the contract changes since the last release and writes a clean.
Agent reviews model-license fit and suggests compliant swaps on the PR
When a PR adds a Hugging Face model, an agent reads the model card and license, judges fit against your commercial-use policy.
Upgrade Impact Router to Module Code Owners
Maps a dependency-bump PR's affected modules to their CODEOWNERS, then DMs each owner on Slack with only the changelog slice that touches code they own.
Re-Voice IVR Prompts on Phone-Tree Config Merge
When a phone-tree config change merges in GitHub, regenerates the ElevenLabs audio for any prompt whose script changed in the diff and opens a follow-up PR adding the new audio…
Upstream Release to Notion Upgrade Brief
When a watched package publishes a new release, fetches the release notes, maps them to the internal modules that depend on it.
Run it inside a business
This workflow drops into a full company template. Import the org, and this is one of the playbooks its agents run.
Run this workflow in your colony.
14-day trial. No DevOps. No Sales call. Provisioned in under a minute.