AI AGENTS
Sentry Repro Attempt Logger with Confidence Scoring
On each new Sentry issue, an agent attempts reproduction, scores its confidence, logs every attempt to Postgres for analytics.
How it runs
The automated pipeline, trigger to output.
- TriggerSentry new-issue alert firesSentry
- ActionFetch trace and run timed shell reproShell
- ActionCompute repro confidence score
- ActionLog attempt record to PostgresPostgres
- LogicOpen MR only above confidence threshold
- OutputWrite failing test and open GitLab MRGitLab
What it does
Instruments the auto-reproduction pipeline itself. For every new Sentry issue, the agent runs a repro attempt, assigns a confidence score, and writes a structured record (issue, attempt result, score, duration) to Postgres so you can measure repro success rate over time. It opens a GitLab MR with a failing test only when the confidence score clears your configured threshold.
When to use it
Use it when you are rolling out auto-reproduction and need data on how often it works before trusting it to file MRs automatically, plus an audit trail of every attempt.
How it works
- 1A Sentry alert fires on a newly created issue.
- 2The agent fetches the trace and runs a shell repro attempt, timing it.
- 3It computes a confidence score from the repro outcome and signal match.
- 4The agent logs the full attempt record to a Postgres analytics table.
- 5Logic gate: open an MR only if the confidence score clears the threshold.
- 6On a high-confidence repro it writes the failing test and opens a GitLab MR.
Set it up
What you configure once, before turning it on.
- 1Connect SentryErrors, performance, releases.
- 2Connect ShellRun sandboxed commands inside the workspace.
- 3Connect PostgresAny Postgres URL — query, write, migrate.
- 4Connect GitLabRepos, MRs, pipelines, registry.
- 5Set each agent's modelWe leave models unset so you pick the tier — fast + cheap, or top-quality.
- 6Tune it to your dataEdit the prompts, filters, and field mappings so it matches how your team works.
- 7Test, then turn it onRun once against a sample, confirm the output, then enable the trigger.
More AI Agents workflows
Custom Metrics Cardinality Spike Pager
A webhook from a Datadog monitor fires when custom-metric cardinality jumps; an agent pinpoints the offending metric and tag, estimates the added cost.
Sentry-to-Confluence Runbook Updater
When a Sentry issue is resolved, the agent finds the matching Confluence runbook page and proposes an inline update with the verified fix.
Stale Doc-PR Chaser for Runbook Gaps
On a daily schedule the agent finds runbook doc PRs that were opened from resolved incidents but never reviewed, summarizes what each one fixes.
Resolved Incident to Public Troubleshooting Doc
For customer-facing errors resolved in Sentry, the agent drafts a sanitized troubleshooting entry and opens a PR to your ReadMe documentation.
On-Call Runbook Gap Closer: Resolved Sentry Issues to Doc PRs
An agent reads each newly resolved Sentry issue, compares the actual fix against your existing runbook, and opens a GitHub PR adding the missing remediation steps.
Weekly On-Call Doc-Gap Digest
Each week the agent reviews every Sentry issue resolved in the last 7 days, ranks the ones whose runbook coverage is missing or thin.
Run it inside a business
This workflow drops into a full company template. Import the org, and this is one of the playbooks its agents run.
Run this workflow in your colony.
14-day trial. No DevOps. No Sales call. Provisioned in under a minute.