AI AGENTS
A/B Experiment Decision Memo Writer to Notion
On demand, an agent reads experiment results from BigQuery, makes a ship/kill/iterate call, and writes a structured decision memo — hypothesis, result, verdict, next step.
How it runs
The automated pipeline, trigger to output.
- TriggerOperator runs with experiment ID (manual)
- ActionPull variant metrics from BigQueryBigQuery
- LogicDecide ship / kill / iterate
- ActionDraft structured decision memo
- OutputCreate memo page in Notion logNotion
What it does
Turns raw experiment numbers into a written, reviewable decision memo. The agent reads the results, reaches a verdict, and drafts a structured memo capturing the hypothesis, what shipped, the measured impact, the decision, and the recommended next action — then files it in your Notion experiment log.
When to use it
Use this when your team needs a durable paper trail of every experiment decision, not just a Slack ping. Ideal for growth and product teams that review past tests quarterly and need the rationale preserved.
How it works
- 1An operator triggers the run manually and supplies the experiment ID.
- 2A BigQuery action pulls the variant metrics and confidence intervals.
- 3The agent reasons over the data to decide ship, kill, or iterate.
- 4The agent composes the memo: hypothesis recap, observed result, verdict with confidence, and a concrete next step.
- 5A Notion action creates a new page in the experiment database with the memo and tagged decision status.
Set it up
What you configure once, before turning it on.
- 1Connect BigQueryDatasets, queries, schemas.
- 2Connect NotionPages, databases, comments.
- 3Set each agent's modelWe leave models unset so you pick the tier — fast + cheap, or top-quality.
- 4Tune it to your dataEdit the prompts, filters, and field mappings so it matches how your team works.
- 5Test, then turn it onRun once against a sample, confirm the output, then enable the trigger.
More AI Agents workflows
Custom Metrics Cardinality Spike Pager
A webhook from a Datadog monitor fires when custom-metric cardinality jumps; an agent pinpoints the offending metric and tag, estimates the added cost.
Sentry-to-Confluence Runbook Updater
When a Sentry issue is resolved, the agent finds the matching Confluence runbook page and proposes an inline update with the verified fix.
Stale Doc-PR Chaser for Runbook Gaps
On a daily schedule the agent finds runbook doc PRs that were opened from resolved incidents but never reviewed, summarizes what each one fixes.
Resolved Incident to Public Troubleshooting Doc
For customer-facing errors resolved in Sentry, the agent drafts a sanitized troubleshooting entry and opens a PR to your ReadMe documentation.
On-Call Runbook Gap Closer: Resolved Sentry Issues to Doc PRs
An agent reads each newly resolved Sentry issue, compares the actual fix against your existing runbook, and opens a GitHub PR adding the missing remediation steps.
Weekly On-Call Doc-Gap Digest
Each week the agent reviews every Sentry issue resolved in the last 7 days, ranks the ones whose runbook coverage is missing or thin.
Run it inside a business
This workflow drops into a full company template. Import the org, and this is one of the playbooks its agents run.
Run this workflow in your colony.
14-day trial. No DevOps. No Sales call. Provisioned in under a minute.