MARKETING

Nightly Reconciled UTM Snapshot to Warehouse

Nightly, pulls all Stripe revenue events with UTM tags, normalizes them against the spend table, and writes a clean reconciled fact table back into BigQuery for BI dashboards.

CategoryMarketing
Enginesim
Difficultyadvanced
Triggerschedule
Steps6
Setup~25 min

How it runs

The automated pipeline, trigger to output.

  • TriggerNightly schedule
  • ActionPull prior-day Stripe revenue with UTM tagsStripeStripe
  • LogicNormalize tags, aggregate per campaign/day
  • ActionJoin spend rows for the dateGoogle BigQueryBigQuery
  • ActionWrite reconciled fact table partitionGoogle BigQueryBigQuery
  • OutputSlack confirmation with row countsSlack

What it does

It maintains the reconciled dataset everything else reads from. Each night it ingests Stripe revenue events, normalizes their UTM tags, joins them to the canonical spend table, and writes a tidy per-campaign-per-day fact table back into BigQuery so dashboards and other workflows query one trusted source.

When to use it

Use it as the foundational pipeline when multiple teams build Looker/Metabase dashboards on top of campaign data and you want reconciliation done once, server-side, instead of in every BI query. Run the other templates downstream of this one.

How it works

  1. 1A nightly schedule triggers the job.
  2. 2Stripe revenue events from the prior day are pulled with their UTM metadata.
  3. 3A logic step normalizes tag casing and aliases, then aggregates revenue per campaign per day.
  4. 4BigQuery supplies the matching daily spend rows for the join key.
  5. 5The reconciled rows (spend, revenue, ROAS, source counts) are written into a dedicated BigQuery fact table, replacing the day's partition idempotently.
  6. 6A Slack confirmation reports row counts and any tags that failed to normalize.

Set it up

What you configure once, before turning it on.

  1. 1
    Connect StripeCustomers, subscriptions, payments.
  2. 2
    Connect BigQueryDatasets, queries, schemas.
  3. 3
    Connect SlackChannels, DMs, threads, mentions.
  4. 4
    Set each agent's modelWe leave models unset so you pick the tier — fast + cheap, or top-quality.
  5. 5
    Tune it to your dataEdit the prompts, filters, and field mappings so it matches how your team works.
  6. 6
    Test, then turn it onRun once against a sample, confirm the output, then enable the trigger.

Run this workflow in your colony.

14-day trial. No DevOps. No Sales call. Provisioned in under a minute.