ENGINEERING
Triage flaky tests from uploaded JUnit results
On every JUnit XML test report posted to the gateway, this diffs results against recent history to spot tests flipping outcomes, quarantines them in Postgres.
How it runs
The automated pipeline, trigger to output.
- TriggerJUnit XML report posted to webhookHTTP webhook
- ActionWrite per-test outcomes to Postgres historyPostgres
- LogicFlag tests flipping pass/fail across runs
- ActionMark flippers quarantined in PostgresPostgres
- OutputPost triage digest to SlackSlack
What it does
It ingests a JUnit XML report whenever CI uploads one, compares each test's outcome to its recent history stored in Postgres, and identifies tests that have flipped between pass and fail without a code change to the test. Those flippers are marked quarantined in the history table and surfaced to the team in a single Slack triage digest.
When to use it
Use it when your CI emits standard JUnit XML and you want a pipeline-agnostic flake detector that works regardless of which runner or language produced the results.
How it works
- 1An HTTP webhook receives the JUnit XML report payload after a CI run.
- 2Results are parsed and each test's current outcome is written to a Postgres history table.
- 3A logic step flags tests whose last N outcomes mix pass and fail (outcome-flipping) as flaky.
- 4Flagged tests get a `quarantined` flag and reason set in Postgres for downstream gating.
- 5A Slack message posts the triage summary — newly quarantined tests, flip counts, and owners — to the team channel.
Set it up
What you configure once, before turning it on.
- 1Connect HTTP webhookTrigger any URL on agent actions.
- 2Connect PostgresAny Postgres URL — query, write, migrate.
- 3Connect SlackChannels, DMs, threads, mentions.
- 4Set each agent's modelWe leave models unset so you pick the tier — fast + cheap, or top-quality.
- 5Tune it to your dataEdit the prompts, filters, and field mappings so it matches how your team works.
- 6Test, then turn it onRun once against a sample, confirm the output, then enable the trigger.
More Engineering workflows
Gate breaking API PRs behind downstream consumer acknowledgement
When a PR introduces a breaking contract change, comments the impact summary back on the PR, applies a blocking label.
Publish a versioned API changelog to Confluence on each release tag
On a new semver release tag, gathers the contract changes since the last release and writes a clean.
Agent reviews model-license fit and suggests compliant swaps on the PR
When a PR adds a Hugging Face model, an agent reads the model card and license, judges fit against your commercial-use policy.
Upgrade Impact Router to Module Code Owners
Maps a dependency-bump PR's affected modules to their CODEOWNERS, then DMs each owner on Slack with only the changelog slice that touches code they own.
Re-Voice IVR Prompts on Phone-Tree Config Merge
When a phone-tree config change merges in GitHub, regenerates the ElevenLabs audio for any prompt whose script changed in the diff and opens a follow-up PR adding the new audio…
Upstream Release to Notion Upgrade Brief
When a watched package publishes a new release, fetches the release notes, maps them to the internal modules that depend on it.
Run it inside a business
This workflow drops into a full company template. Import the org, and this is one of the playbooks its agents run.
Run this workflow in your colony.
14-day trial. No DevOps. No Sales call. Provisioned in under a minute.