ENGINEERING

Weekly BigQuery top-offenders report to Confluence with trend deltas

Every week, ranks the queries that consumed the most BigQuery bytes and the ones whose cost grew fastest week-over-week, then publishes a Confluence page with the leaderboard.

CategoryEngineering
Enginesim
Difficultyintermediate
Triggerschedule
Steps5
Setup~15 min

How it runs

The automated pipeline, trigger to output.

  • TriggerWeekly schedule
  • ActionAggregate this week vs last week by query hashGoogle BigQueryBigQuery
  • LogicRank top consumers and fastest climbers
  • ActionGather plan summaries for top offendersGoogle BigQueryBigQuery
  • OutputPublish Confluence top-offenders reportConfluenceConfluence

What it does

This produces a recurring, shareable digest rather than per-incident tickets. It rolls up a week of BigQuery usage, ranks the heaviest queries and the steepest week-over-week climbers, and writes a Confluence page leadership and the data platform team can review together. Each entry carries a plan summary so the report doubles as a tuning backlog.

When to use it

Use for weekly platform reviews where you want a single living document of where BigQuery spend goes and which queries are trending the wrong way. Pairs well with the real-time and daily hunters for after-the-fact accountability.

How it works

  1. 1A weekly schedule kicks off the rollup.
  2. 2BigQuery job history is aggregated by query hash for this week and last week.
  3. 3A logic step ranks top consumers and computes week-over-week cost deltas, flagging the fastest climbers.
  4. 4Plan summaries are gathered for the top offenders.
  5. 5A Confluence page is published with the leaderboard, deltas, and plan notes as the final output.

Set it up

What you configure once, before turning it on.

  1. 1
    Connect BigQueryDatasets, queries, schemas.
  2. 2
    Connect ConfluenceSpaces, pages, blueprints.
  3. 3
    Set each agent's modelWe leave models unset so you pick the tier — fast + cheap, or top-quality.
  4. 4
    Tune it to your dataEdit the prompts, filters, and field mappings so it matches how your team works.
  5. 5
    Test, then turn it onRun once against a sample, confirm the output, then enable the trigger.

Run this workflow in your colony.

14-day trial. No DevOps. No Sales call. Provisioned in under a minute.