DATA OPS
Log Every BigQuery Schema Change to a Snowflake Audit Trail
Triggered when a BigQuery table schema changes, it captures the before/after column diff and writes a structured row into a Snowflake audit table.
How it runs
The automated pipeline, trigger to output.
- TriggerBigQuery schema-change eventBigQuery
- ActionRead updated BigQuery schemaBigQuery
- LogicDiff against prior recorded state
- ActionInsert change row into Snowflake audit tableSnowflake
- OutputUpdated queryable schema-history logSnowflake
What it does
This workflow turns ephemeral schema changes into a permanent, queryable record. When a BigQuery table's schema changes, it computes the diff against the last known state and inserts a structured audit row into a dedicated Snowflake history table: what table changed, which columns, the old and new definitions, and when.
When to use it
Use it when you need an answer to "when did this column change and to what?" for compliance, debugging downstream breakages, or post-mortems. Instead of guessing from `INFORMATION_SCHEMA` snapshots, you query a real audit trail.
How it works
- 1A schema-change event for a BigQuery table triggers the workflow.
- 2It reads the new BigQuery schema for the affected table.
- 3A logic step diffs it against the prior recorded state to produce a change record.
- 4It inserts the structured change record as a row in the Snowflake audit table.
- 5The audit table is now updated and queryable as the canonical schema-history log.
Set it up
What you configure once, before turning it on.
- 1Connect BigQueryDatasets, queries, schemas.
- 2Connect SnowflakeWarehouses, queries, shares.
- 3Set each agent's modelWe leave models unset so you pick the tier — fast + cheap, or top-quality.
- 4Tune it to your dataEdit the prompts, filters, and field mappings so it matches how your team works.
- 5Test, then turn it onRun once against a sample, confirm the output, then enable the trigger.
More Data Ops workflows
BigQuery Per-Team Budget Breach Alert to PagerDuty
Tracks month-to-date BigQuery scheduled-query spend per team and, when a team crosses its monthly budget, pages the team's on-call in PagerDuty and snapshots the spend breakdown…
dbt orphan model detector with Linear cleanup tickets
Scans your dbt manifest for models that no other model, exposure, or BI tool consumes.
Weekly BigQuery Cost Trend Sheet and Exec Digest
Compiles week-over-week BigQuery scheduled-query cost by owner and dataset into a Google Sheet with trend columns.
Backfill Missing Owner Labels on BigQuery Scheduled Queries
Finds scheduled queries with no owner label, infers the likely owner from creator metadata and target-table lineage, proposes a label.
Daily BigQuery Scheduled-Query Cost Attribution to Owners
Each morning, totals the prior day's on-demand bytes-billed per scheduled query, maps each query to its owner from a label, and posts a per-owner cost leaderboard to Slack.
dbt source freshness watcher with severity-routed alerts
Checks Snowflake loaded-at timestamps against each dbt source's freshness SLA, then routes warnings to Slack and hard breaches to a PagerDuty incident so stale data never…
Run it inside a business
This workflow drops into a full company template. Import the org, and this is one of the playbooks its agents run.
Run this workflow in your colony.
14-day trial. No DevOps. No Sales call. Provisioned in under a minute.