DATA OPS
BigQuery Team Budget Breach Detector with PagerDuty Escalation
Hourly, accumulates each team's month-to-date BigQuery spend against budgets stored in Airtable.
How it runs
The automated pipeline, trigger to output.
- TriggerHourly schedule
- ActionCompute month-to-date spend per team in BigQueryBigQuery
- ActionRead per-team monthly budgets from AirtableAirtable
- LogicCompute percent of budget, assign warn/breach tier
- ActionSend 90% Slack warning to team ownerSlack
- OutputOpen PagerDuty incident for over-budget teamsPagerDuty
What it does
It watches month-to-date warehouse spend per team against per-team monthly budgets held in Airtable. Once an hour it recomputes MTD spend, compares it to budget, and escalates by severity: a Slack heads-up at 90 percent, and a PagerDuty incident at 100 percent so a runaway team can be throttled before the bill balloons.
When to use it
Use it when teams have committed warehouse budgets and an overrun has real financial consequence. Hourly checks catch a misbehaving backfill or scheduled query mid-day instead of at month-end, and the PagerDuty page ensures someone actually acts.
How it works
- 1An hourly schedule fires.
- 2BigQuery sums month-to-date billed cost grouped by user, joined in-query to a team via a mapping table.
- 3Airtable supplies each team's monthly budget.
- 4A logic step computes percent-of-budget per team and assigns a tier (under 90, 90-100, over 100).
- 5Teams in the 90-100 tier get a Slack warning to their owner.
- 6Teams over 100 percent trigger a PagerDuty incident routed to the data on-call.
Set it up
What you configure once, before turning it on.
- 1Connect BigQueryDatasets, queries, schemas.
- 2Connect AirtableBases, tables, views, automations.
- 3Connect SlackChannels, DMs, threads, mentions.
- 4Connect PagerDutyIncidents, on-call, escalations.
- 5Set each agent's modelWe leave models unset so you pick the tier — fast + cheap, or top-quality.
- 6Tune it to your dataEdit the prompts, filters, and field mappings so it matches how your team works.
- 7Test, then turn it onRun once against a sample, confirm the output, then enable the trigger.
More Data Ops workflows
Snowflake column type-drift sentinel with Linear fix ticket
Snapshots the data types of every column in your tracked Snowflake schemas on a schedule, diffs against the last snapshot.
Daily BigQuery Scheduled-Query Cost Attribution to Owners
Each morning, totals the prior day's on-demand bytes-billed per scheduled query, maps each query to its owner from a label, and posts a per-owner cost leaderboard to Slack.
BigQuery dropped/renamed column sentinel with PagerDuty incident
Detects when a column is dropped or renamed in your governed BigQuery datasets and, because that breaks downstream queries hard, pages the on-call via PagerDuty and posts…
PR-time Snowflake schema contract check on dbt model changes
When a pull request changes a dbt model, it compares the model's declared output columns against the live Snowflake table it will replace and blocks the merge with a GitHub check…
Agent-triaged warehouse drift with impact analysis and runbook update
On a webhook from your warehouse audit log, an agent investigates the changed column, traces which downstream models and dashboards depend on it.
Cross-warehouse replication schema mismatch reconciler
Compares the column shape of mirrored tables between BigQuery and Snowflake and, when a replicated table has drifted out of sync between the two, opens an Asana task for the data…
Run it inside a business
This workflow drops into a full company template. Import the org, and this is one of the playbooks its agents run.
Run this workflow in your colony.
14-day trial. No DevOps. No Sales call. Provisioned in under a minute.