AI AGENTS
Replicate Version-Drift Watcher with Golden-Prompt Migration Ticket
An agent watches your pinned Replicate model versions, and when one is deprecated it dry-runs the successor version against your golden prompt set and opens a GitLab migration…
How it runs
The automated pipeline, trigger to output.
- TriggerDaily schedule fires the drift check
- ActionList pinned Replicate versions and check deprecation statusReplicate
- LogicFilter to newly deprecated versions; exit if none
- ActionDry-run golden prompts on old and successor versionsReplicate
- LogicDiff paired outputs and score regression drift
- ActionOpen GitLab migration issue with embedded diffsGitLab
- OutputPost Slack summary linking the ticketSlack
What it does
This agent guards every Replicate inference endpoint your app pins to a specific model version. When Replicate marks a pinned version deprecated, the agent automatically benchmarks the recommended successor version against a saved set of golden prompts, diffs the outputs, and files a GitLab issue so an engineer can approve or reject the migration with real evidence in hand.
When to use it
Use it when your production code hard-pins Replicate version hashes and a silent deprecation could break inference weeks later. It turns a surprise outage into a reviewed, evidence-backed pull-forward.
How it works
- 1On a daily schedule, the agent lists your pinned Replicate versions and checks each model's status.
- 2A logic step filters to versions newly flagged deprecated; if none, the run ends quietly.
- 3For each deprecated version, the agent runs every golden prompt on both the old and the successor version.
- 4It diffs the paired outputs, scoring semantic drift and flagging regressions.
- 5It opens a GitLab issue titled with the model, embeds the side-by-side diffs, and assigns the owning team.
- 6It posts a Slack summary linking the ticket and the highest-drift prompts.
Set it up
What you configure once, before turning it on.
- 1Connect ReplicateImage, video, and model inference.
- 2Connect GitLabRepos, MRs, pipelines, registry.
- 3Connect SlackChannels, DMs, threads, mentions.
- 4Set each agent's modelWe leave models unset so you pick the tier — fast + cheap, or top-quality.
- 5Tune it to your dataEdit the prompts, filters, and field mappings so it matches how your team works.
- 6Test, then turn it onRun once against a sample, confirm the output, then enable the trigger.
More AI Agents workflows
Custom Metrics Cardinality Spike Pager
A webhook from a Datadog monitor fires when custom-metric cardinality jumps; an agent pinpoints the offending metric and tag, estimates the added cost.
Sentry-to-Confluence Runbook Updater
When a Sentry issue is resolved, the agent finds the matching Confluence runbook page and proposes an inline update with the verified fix.
Stale Doc-PR Chaser for Runbook Gaps
On a daily schedule the agent finds runbook doc PRs that were opened from resolved incidents but never reviewed, summarizes what each one fixes.
Resolved Incident to Public Troubleshooting Doc
For customer-facing errors resolved in Sentry, the agent drafts a sanitized troubleshooting entry and opens a PR to your ReadMe documentation.
On-Call Runbook Gap Closer: Resolved Sentry Issues to Doc PRs
An agent reads each newly resolved Sentry issue, compares the actual fix against your existing runbook, and opens a GitHub PR adding the missing remediation steps.
Weekly On-Call Doc-Gap Digest
Each week the agent reviews every Sentry issue resolved in the last 7 days, ranks the ones whose runbook coverage is missing or thin.
Run it inside a business
This workflow drops into a full company template. Import the org, and this is one of the playbooks its agents run.
Run this workflow in your colony.
14-day trial. No DevOps. No Sales call. Provisioned in under a minute.