DEVOPS

Catch BigQuery scheduled-query cost regressions and file a tuning ticket

Daily compares each scheduled query's bytes billed to its 7-day baseline and, when a job's cost jumps past the regression threshold, opens an Asana tuning task and alerts Slack.

CategoryDevOps
Enginesim
Difficultyadvanced
Triggerschedule
Steps5
Setup~25 min

How it runs

The automated pipeline, trigger to output.

  • TriggerDaily schedule
  • ActionRead per-job bytes billed for yesterday and prior 7 daysGoogle BigQueryBigQuery
  • LogicCompute baseline and flag cost regressions
  • ActionCreate Asana tuning task per regressionAsanaAsana
  • OutputPost regression summary to SlackSlack

What it does

It watches your recurring BigQuery scheduled queries for cost creep. Each day it pulls per-job bytes billed, compares every scheduled query against its own trailing 7-day average, and flags any job whose cost jumped beyond a regression threshold, usually because an underlying table grew or a filter broke. Each regression becomes an Asana task and a Slack alert.

When to use it

Use it when scheduled queries are your main spend and a single one quietly doubling in cost would otherwise go unnoticed for weeks. Baseline comparison catches relative regressions that a flat budget threshold misses.

How it works

  1. 1A daily schedule triggers the flow.
  2. 2A BigQuery query reads per-scheduled-job bytes billed for yesterday and the prior 7 days.
  3. 3A logic step computes each job's baseline and flags jobs over the regression threshold.
  4. 4For each regression, an Asana task is created with the job name, baseline, and new cost.
  5. 5A summary of regressions is posted to Slack for the data platform team.

Set it up

What you configure once, before turning it on.

  1. 1
    Connect BigQueryDatasets, queries, schemas.
  2. 2
    Connect AsanaTasks, projects, milestones — everywhere.
  3. 3
    Connect SlackChannels, DMs, threads, mentions.
  4. 4
    Set each agent's modelWe leave models unset so you pick the tier — fast + cheap, or top-quality.
  5. 5
    Tune it to your dataEdit the prompts, filters, and field mappings so it matches how your team works.
  6. 6
    Test, then turn it onRun once against a sample, confirm the output, then enable the trigger.

Run this workflow in your colony.

14-day trial. No DevOps. No Sales call. Provisioned in under a minute.