ENGINEERING
Dependency Canary: Honeycomb-Triggered Rollback and PagerDuty Escalation
Continuously watches a deployed canary's Honeycomb latency and error SLOs and, on a hard breach, reverts the dependency PR to draft, files a PagerDuty incident.
How it runs
The automated pipeline, trigger to output.
- TriggerHoneycomb SLO burn alert (hard breach)Honeycomb
- LogicMatch alert to open canary PR
- ActionRevert PR to draft, remove from merge queueGitHub
- ActionOpen PagerDuty incidentPagerDuty
- OutputPost rollback context to SlackSlack
What it does
Acts as the safety net for a live canary. It polls Honeycomb against your SLO thresholds and, the moment a dependency bump breaches them, it pulls the PR out of the merge queue, opens a PagerDuty incident, and broadcasts the rollback context so responders move fast.
When to use it
Use it for high-traffic services where a bad upgrade needs an automated stop, not a human noticing a dashboard. This is the escalation layer that complements the gentler comment-only canary checks.
How it works
- 1A Honeycomb trigger fires when a canary SLO burn alert crosses the hard threshold.
- 2A logic step confirms the alert maps to a service tied to an open canary PR.
- 3It reverts that PR to draft and removes it from the merge queue via GitHub.
- 4It opens a PagerDuty incident with the SLO breach summary and Honeycomb link.
- 5It posts the incident, PR, and metric snapshot to the Slack incident channel.
Set it up
What you configure once, before turning it on.
- 1Connect HoneycombDistributed traces and queries.
- 2Connect GitHubRepos, issues, pull requests, actions.
- 3Connect PagerDutyIncidents, on-call, escalations.
- 4Connect SlackChannels, DMs, threads, mentions.
- 5Set each agent's modelWe leave models unset so you pick the tier — fast + cheap, or top-quality.
- 6Tune it to your dataEdit the prompts, filters, and field mappings so it matches how your team works.
- 7Test, then turn it onRun once against a sample, confirm the output, then enable the trigger.
More Engineering workflows
Agent reviews model-license fit and suggests compliant swaps on the PR
When a PR adds a Hugging Face model, an agent reads the model card and license, judges fit against your commercial-use policy.
Block PRs that add incompatible Hugging Face model licenses
When a pull request adds or bumps a Hugging Face model dependency, it fetches the model card license, checks it against your org's allowed-license policy.
Quarterly Logging Hygiene Audit Agent
An agent-driven quarterly sweep that surveys all Axiom datasets, builds a logging-hygiene scorecard per service.
Post-Merge Log Volume Recheck After Downsampling PR
After a log-level PR merges, waits a day then re-queries Axiom to confirm the targeted stream's volume actually dropped.
Axiom Ingest Cost Spike to Linear Triage Ticket
When Axiom ingest volume spikes beyond its baseline, identifies which service caused it and files a Linear ticket with the offending log stream, sample lines, and a downsampling…
File a Linear license-review ticket for risky model adds
When a PR introduces a Hugging Face model with a non-permissive or unknown license, it opens a Linear issue assigned to the legal-review team with the model, license.
Run it inside a business
This workflow drops into a full company template. Import the org, and this is one of the playbooks its agents run.
Run this workflow in your colony.
14-day trial. No DevOps. No Sales call. Provisioned in under a minute.