DATA OPS

Daily BigQuery Scheduled-Query Health Digest

Runs every morning, scans the last 24 hours of BigQuery scheduled-query runs, and posts a Discord digest of failures, retries, and chronic flaky queries.

CategoryData Ops
Enginesim
Difficultyintermediate
Triggerschedule
Steps5
Setup~15 min

How it runs

The automated pipeline, trigger to output.

  • TriggerDaily morning schedule
  • ActionPull 24h scheduled-query run historyGoogle BigQueryBigQuery
  • LogicAggregate counts, flag chronic offenders
  • ActionOpen deduped Trello card per offenderTrelloTrello
  • OutputPost health digest to DiscordDiscordDiscord

What it does

Gives data teams a once-a-day rollup of every scheduled query's health instead of a stream of individual alerts. It counts failures and successes over the trailing day, flags queries that failed more than once, and distinguishes one-off blips from chronically flaky configs that need real attention.

When to use it

Use it when you run dozens of scheduled queries and want trend visibility rather than per-event noise. It is the calm morning report that tells you what quietly broke overnight and which pipelines are degrading over time.

How it works

  1. 1A daily schedule trigger fires each morning before standup.
  2. 2An action pulls the run history for all scheduled queries in the project over the last 24 hours.
  3. 3A logic step aggregates per-query success and failure counts and marks any query with repeated failures as a chronic offender.
  4. 4For each chronic offender it opens a Trello card (deduplicated against existing open cards) so it does not get lost in the digest.
  5. 5It posts a formatted Discord digest summarizing total runs, failures, recoveries, and the offender list.

Set it up

What you configure once, before turning it on.

  1. 1
    Connect BigQueryDatasets, queries, schemas.
  2. 2
    Connect DiscordCommunity channels + voice + bots.
  3. 3
    Connect TrelloKanban boards for everything.
  4. 4
    Set each agent's modelWe leave models unset so you pick the tier — fast + cheap, or top-quality.
  5. 5
    Tune it to your dataEdit the prompts, filters, and field mappings so it matches how your team works.
  6. 6
    Test, then turn it onRun once against a sample, confirm the output, then enable the trigger.

Run this workflow in your colony.

14-day trial. No DevOps. No Sales call. Provisioned in under a minute.