DATA OPS
Agent-Driven CSV Schema Drift Triage and Partner Outreach
When a partner CSV fails validation, an agent diagnoses whether the failures are schema drift or dirty data, files a Linear issue with the analysis.
How it runs
The automated pipeline, trigger to output.
- TriggerNew partner CSV lands in S3AWS S3
- LogicValidate rows and build defect report
- ActionAgent classifies schema drift vs dirty dataOpenAI
- ActionFile Linear issue for schema drift casesLinear
- ActionDraft and send partner correction emailGmail
- OutputPost triage summary to SlackSlack
What it does
This workflow validates an incoming partner CSV, and when rows fail, hands the defect report to an agent that reasons about the failures: is the partner sending a renamed column (schema drift) or just bad values in the right shape (dirty data)? The agent classifies the root cause, opens a Linear issue for engineering when it's drift, and drafts a clear correction email to the partner.
When to use it
Use it when triaging feed failures eats analyst time and the same diagnosis gets repeated by hand. The agent does the first-pass investigation and produces both an internal ticket and an outbound message, so a human only reviews and sends.
How it works
- 1A new partner CSV in the S3 prefix fires the trigger.
- 2Rows are validated against the contract schema and a defect report is built.
- 3If there are failures, the agent analyzes the defects to separate schema drift from dirty data.
- 4For schema drift, the agent files a Linear issue with the offending columns and a proposed fix.
- 5The agent drafts a partner-ready correction email and sends it via Gmail for review.
- 6A summary of the triage is posted to Slack.
Set it up
What you configure once, before turning it on.
- 1Connect AWS S3Buckets, objects, signed URLs.
- 2Connect LinearIssues, projects, cycles, triage.
- 3Connect GmailRead, draft, send, label.
- 4Connect SlackChannels, DMs, threads, mentions.
- 5Connect OpenAIModels, embeddings, files.
- 6Set each agent's modelWe leave models unset so you pick the tier — fast + cheap, or top-quality.
- 7Tune it to your dataEdit the prompts, filters, and field mappings so it matches how your team works.
- 8Test, then turn it onRun once against a sample, confirm the output, then enable the trigger.
More Data Ops workflows
Snowflake column type-drift sentinel with Linear fix ticket
Snapshots the data types of every column in your tracked Snowflake schemas on a schedule, diffs against the last snapshot.
Daily BigQuery Scheduled-Query Cost Attribution to Owners
Each morning, totals the prior day's on-demand bytes-billed per scheduled query, maps each query to its owner from a label, and posts a per-owner cost leaderboard to Slack.
BigQuery dropped/renamed column sentinel with PagerDuty incident
Detects when a column is dropped or renamed in your governed BigQuery datasets and, because that breaks downstream queries hard, pages the on-call via PagerDuty and posts…
PR-time Snowflake schema contract check on dbt model changes
When a pull request changes a dbt model, it compares the model's declared output columns against the live Snowflake table it will replace and blocks the merge with a GitHub check…
Agent-triaged warehouse drift with impact analysis and runbook update
On a webhook from your warehouse audit log, an agent investigates the changed column, traces which downstream models and dashboards depend on it.
Cross-warehouse replication schema mismatch reconciler
Compares the column shape of mirrored tables between BigQuery and Snowflake and, when a replicated table has drifted out of sync between the two, opens an Asana task for the data…
Run it inside a business
This workflow drops into a full company template. Import the org, and this is one of the playbooks its agents run.
Run this workflow in your colony.
14-day trial. No DevOps. No Sales call. Provisioned in under a minute.