ENGINEERING
Weekly Flaky-Test Scorecard to Confluence with Per-Owner Linear Tickets
Every Monday aggregates the week's flaky-test data from GitHub and Honeycomb, publishes a ranked scorecard to Confluence.
How it runs
The automated pipeline, trigger to output.
- TriggerWeekly Monday schedule fires
- ActionCollect flake data from GitHub and HoneycombGitHub
- LogicRank by flake rate and flag stale unresolved flakes
- ActionPublish ranked scorecard to ConfluenceConfluence
- ActionFile owner-assigned tickets for stale flakesLinear
- OutputPost scorecard link and escalations to SlackSlack
What it does
This workflow gives engineering leadership a recurring view of test-suite health. Each week it pulls flaky-test occurrences from GitHub checks and Honeycomb telemetry, ranks tests by flake rate and trend, and publishes a scorecard page to Confluence. Any flaky test still unresolved after two weeks gets a fresh, owner-assigned Linear ticket so chronic offenders do not fade into the backlog.
When to use it
Use this when you need a visible, accountable cadence around flaky tests, a single linkable scorecard for standups and reviews, plus automatic escalation of stale flakes to their owners.
How it works
- 1A weekly schedule triggers the run on Monday morning.
- 2The flow collects the week's flake occurrences from GitHub and Honeycomb and merges them by test.
- 3A logic step ranks tests by flake rate and week-over-week trend, and flags any unresolved beyond two weeks.
- 4A Confluence page is created or updated with the ranked scorecard.
- 5For each stale flake, an owner-assigned Linear ticket is filed.
- 6A Slack post links the scorecard and lists the escalated tests.
Set it up
What you configure once, before turning it on.
- 1Connect GitHubRepos, issues, pull requests, actions.
- 2Connect HoneycombDistributed traces and queries.
- 3Connect ConfluenceSpaces, pages, blueprints.
- 4Connect LinearIssues, projects, cycles, triage.
- 5Connect SlackChannels, DMs, threads, mentions.
- 6Set each agent's modelWe leave models unset so you pick the tier — fast + cheap, or top-quality.
- 7Tune it to your dataEdit the prompts, filters, and field mappings so it matches how your team works.
- 8Test, then turn it onRun once against a sample, confirm the output, then enable the trigger.
More Engineering workflows
Agent reviews model-license fit and suggests compliant swaps on the PR
When a PR adds a Hugging Face model, an agent reads the model card and license, judges fit against your commercial-use policy.
Block PRs that add incompatible Hugging Face model licenses
When a pull request adds or bumps a Hugging Face model dependency, it fetches the model card license, checks it against your org's allowed-license policy.
Quarterly Logging Hygiene Audit Agent
An agent-driven quarterly sweep that surveys all Axiom datasets, builds a logging-hygiene scorecard per service.
Post-Merge Log Volume Recheck After Downsampling PR
After a log-level PR merges, waits a day then re-queries Axiom to confirm the targeted stream's volume actually dropped.
Axiom Ingest Cost Spike to Linear Triage Ticket
When Axiom ingest volume spikes beyond its baseline, identifies which service caused it and files a Linear ticket with the offending log stream, sample lines, and a downsampling…
File a Linear license-review ticket for risky model adds
When a PR introduces a Hugging Face model with a non-permissive or unknown license, it opens a Linear issue assigned to the legal-review team with the model, license.
Run it inside a business
This workflow drops into a full company template. Import the org, and this is one of the playbooks its agents run.
Run this workflow in your colony.
14-day trial. No DevOps. No Sales call. Provisioned in under a minute.