DATA OPS
Agent-driven root-cause triage for late BigQuery partitions
When a partition lands late, an agent investigates the likely cause by querying load-job history and upstream source freshness.
How it runs
The automated pipeline, trigger to output.
- TriggerWebhook delivers breach eventHTTP webhook
- ActionFetch table load-job history and errorsBigQuery
- ActionCheck upstream source table freshnessBigQuery
- LogicAgent drafts ranked root-cause hypothesis
- ActionFile GitHub issue with diagnosisGitHub
- OutputOpen Slack triage thread linking the issueSlack
What it does
Instead of just flagging lateness, this workflow reasons about why. On a breach, an agent pulls the table's recent load-job errors and durations, checks whether upstream source tables were themselves late, and weighs the evidence to produce a ranked root-cause hypothesis (e.g. upstream delay vs failed load vs schema drift) with a concrete next step. It then opens a GitHub issue and starts a Slack triage thread.
When to use it
Use it when the same handful of pipelines break repeatedly and your team wastes the first 20 minutes of every incident re-investigating. Lets the agent do the first-pass diagnosis.
How it works
- 1A webhook trigger delivers the breach event with the affected table.
- 2A BigQuery action fetches recent load-job history, errors, and durations for the table.
- 3A BigQuery action checks freshness of the declared upstream source tables.
- 4An agent reasons over the evidence and drafts a ranked root-cause hypothesis with a recommended action.
- 5A GitHub action files an issue with the diagnosis attached.
- 6A Slack message opens a triage thread linking the issue.
Set it up
What you configure once, before turning it on.
- 1Connect HTTP webhookTrigger any URL on agent actions.
- 2Connect BigQueryDatasets, queries, schemas.
- 3Connect GitHubRepos, issues, pull requests, actions.
- 4Connect SlackChannels, DMs, threads, mentions.
- 5Set each agent's modelWe leave models unset so you pick the tier — fast + cheap, or top-quality.
- 6Tune it to your dataEdit the prompts, filters, and field mappings so it matches how your team works.
- 7Test, then turn it onRun once against a sample, confirm the output, then enable the trigger.
More Data Ops workflows
Snowflake column type-drift sentinel with Linear fix ticket
Snapshots the data types of every column in your tracked Snowflake schemas on a schedule, diffs against the last snapshot.
Daily BigQuery Scheduled-Query Cost Attribution to Owners
Each morning, totals the prior day's on-demand bytes-billed per scheduled query, maps each query to its owner from a label, and posts a per-owner cost leaderboard to Slack.
BigQuery dropped/renamed column sentinel with PagerDuty incident
Detects when a column is dropped or renamed in your governed BigQuery datasets and, because that breaks downstream queries hard, pages the on-call via PagerDuty and posts…
PR-time Snowflake schema contract check on dbt model changes
When a pull request changes a dbt model, it compares the model's declared output columns against the live Snowflake table it will replace and blocks the merge with a GitHub check…
Agent-triaged warehouse drift with impact analysis and runbook update
On a webhook from your warehouse audit log, an agent investigates the changed column, traces which downstream models and dashboards depend on it.
Cross-warehouse replication schema mismatch reconciler
Compares the column shape of mirrored tables between BigQuery and Snowflake and, when a replicated table has drifted out of sync between the two, opens an Asana task for the data…
Run it inside a business
This workflow drops into a full company template. Import the org, and this is one of the playbooks its agents run.
Run this workflow in your colony.
14-day trial. No DevOps. No Sales call. Provisioned in under a minute.