DEVOPS
Alert when a test's flake rate crosses a threshold
Runs on a schedule, computes each test's recent flake rate from Datadog CI Visibility metrics.
How it runs
The automated pipeline, trigger to output.
- TriggerScheduled daily check
- ActionQuery Datadog CI flake ratesDatadog
- LogicFilter tests over threshold, dedupe tracked
- ActionCreate Linear ticket per offenderLinear
- OutputPost daily flake summary to SlackSlack
What it does
This template watches your test suite's reliability over time. On a schedule it queries Datadog CI Visibility for per-test flake rates over a rolling window, finds tests whose flakiness has crossed a threshold you set, and escalates them so they get owned and fixed instead of silently rotting.
When to use it
Use it when you already ship CI test results to Datadog and want a standing watchdog that catches degrading tests early, rather than waiting for them to block a release.
How it works
- 1A schedule trigger fires the check (e.g. every morning).
- 2The flow queries Datadog CI Visibility for flake rate per test across the rolling window.
- 3A logic step filters to tests above the threshold and dedupes ones already tracked.
- 4For each new offender, Linear creates a ticket with the test name, flake rate, and recent failing run links.
- 5A Slack message posts the day's new and worsening flakes to the team channel.
Set it up
What you configure once, before turning it on.
- 1Connect DatadogMetrics, traces, log search.
- 2Connect LinearIssues, projects, cycles, triage.
- 3Connect SlackChannels, DMs, threads, mentions.
- 4Set each agent's modelWe leave models unset so you pick the tier — fast + cheap, or top-quality.
- 5Tune it to your dataEdit the prompts, filters, and field mappings so it matches how your team works.
- 6Test, then turn it onRun once against a sample, confirm the output, then enable the trigger.
More DevOps workflows
Block costly Hugging Face Space hardware upgrades in PR review
When a pull request changes a Space's hardware config, it estimates the new monthly cost and posts a GitHub PR comment that flags upgrades crossing a budget ceiling.
Auto-spin a Zoom war-room when PagerDuty hits SEV-1
When a PagerDuty incident escalates to a critical severity, this workflow creates a dedicated Zoom meeting and posts the bridge link to the incident's Slack channel so responders…
Page on-call when a Hugging Face Space build is stuck or errored
Polls Hugging Face Space runtime status on a schedule and opens a PagerDuty incident when a Space sits in a build or error state past a deadline, with a Slack heads-up.
Slack-approved pause for idle Hugging Face Spaces
On a daily scan it finds idle paid Spaces and posts an interactive Slack approval; on approve it pauses the Space and logs the decision to a GitHub issue audit trail.
Hugging Face Spaces idle-runtime sweep with auto-pause
On a schedule, scans all Hugging Face Spaces for ones running idle past a threshold, pauses them to stop billing, and posts a Slack summary with the estimated monthly savings.
Open a Zoom war-room from a Datadog multi-alert storm
When a Datadog monitor crosses a critical threshold, this workflow dedupes against active incidents, and only for a genuinely new outage it creates a Zoom bridge.
Run it inside a business
This workflow drops into a full company template. Import the org, and this is one of the playbooks its agents run.
Run this workflow in your colony.
14-day trial. No DevOps. No Sales call. Provisioned in under a minute.