ENGINEERING
Nightly flaky-test scanner from GitHub Actions history
Each night, scans recent GitHub Actions runs to find tests that pass and fail nondeterministically on the same commit.
How it runs
The automated pipeline, trigger to output.
- TriggerNightly schedule fires
- ActionFetch recent GitHub Actions runs and test artifactsGitHub
- LogicScore tests with mixed pass/fail on the same commit
- ActionCreate or update a Linear issue per flaky testLinear
- OutputPost new-flake summary to SlackSlack
What it does
Finds the tests that are quietly eroding trust in your CI by failing intermittently. It pulls the last N days of GitHub Actions runs, groups results by test identity across commits, and flags any test that has both passes and failures on the same SHA. Each confirmed flake becomes a tracked Linear issue so it never gets lost in the noise.
When to use it
Run it on any repo where 'just re-run the job' has become a habit. It is the right fit when you want a standing, deduplicated backlog of flaky tests instead of ad-hoc Slack complaints.
How it works
- 1A nightly schedule fires the workflow.
- 2It fetches recent workflow runs and their JUnit/test artifacts from GitHub.
- 3A logic step computes a per-test flake score: tests with mixed pass/fail on identical commits above a threshold are kept.
- 4For each flake it checks GitHub for an existing tracking label to avoid duplicates.
- 5It creates or updates a Linear issue with the failure rate, owning team, and links to the offending runs.
- 6It posts a one-line summary of new flakes to the team Slack channel.
Set it up
What you configure once, before turning it on.
- 1Connect GitHubRepos, issues, pull requests, actions.
- 2Connect LinearIssues, projects, cycles, triage.
- 3Connect SlackChannels, DMs, threads, mentions.
- 4Set each agent's modelWe leave models unset so you pick the tier — fast + cheap, or top-quality.
- 5Tune it to your dataEdit the prompts, filters, and field mappings so it matches how your team works.
- 6Test, then turn it onRun once against a sample, confirm the output, then enable the trigger.
More Engineering workflows
Gate breaking API PRs behind downstream consumer acknowledgement
When a PR introduces a breaking contract change, comments the impact summary back on the PR, applies a blocking label.
Publish a versioned API changelog to Confluence on each release tag
On a new semver release tag, gathers the contract changes since the last release and writes a clean.
Agent reviews model-license fit and suggests compliant swaps on the PR
When a PR adds a Hugging Face model, an agent reads the model card and license, judges fit against your commercial-use policy.
Upgrade Impact Router to Module Code Owners
Maps a dependency-bump PR's affected modules to their CODEOWNERS, then DMs each owner on Slack with only the changelog slice that touches code they own.
Re-Voice IVR Prompts on Phone-Tree Config Merge
When a phone-tree config change merges in GitHub, regenerates the ElevenLabs audio for any prompt whose script changed in the diff and opens a follow-up PR adding the new audio…
Upstream Release to Notion Upgrade Brief
When a watched package publishes a new release, fetches the release notes, maps them to the internal modules that depend on it.
Run it inside a business
This workflow drops into a full company template. Import the org, and this is one of the playbooks its agents run.
Run this workflow in your colony.
14-day trial. No DevOps. No Sales call. Provisioned in under a minute.