MARKET RESEARCH

Trend Term History Loader to BigQuery

Captures the day's Brave Search snapshot for your tracked terms, normalizes and clusters them.

CategoryMarket Research
Enginesim
Difficultyadvanced
Triggerschedule
Steps5
Setup~25 min

How it runs

The automated pipeline, trigger to output.

  • TriggerDaily schedule starts the load
  • ActionPull Brave Search snapshot per tracked termBraveBrave Search
  • ActionNormalize terms and assign cluster idsOpenAI
  • LogicShape records to warehouse row schema
  • OutputAppend rows to BigQuery trend-history tableGoogle BigQueryBigQuery

What it does

This workflow turns daily search signal into durable data. It pulls Brave Search results for every tracked term, has OpenAI normalize and tag each into a theme cluster, and appends timestamped rows to a BigQuery table so analysts can chart emergence curves and run their own SQL over months of trend history.

When to use it

Use it when a one-off brief isn't enough and you need a warehouse-backed dataset — for dashboards, cohort comparisons, or feeding downstream models. Best for data and analytics teams who own a BigQuery instance.

How it works

  1. 1A daily schedule starts the load.
  2. 2Brave Search returns result counts, top domains, and snippets per tracked term.
  3. 3An OpenAI step normalizes raw terms and assigns each to a theme cluster with a stable cluster id.
  4. 4A logic step shapes the records into the warehouse row schema and stamps the run date.
  5. 5A BigQuery step appends the rows to the trend-history table for querying.

Set it up

What you configure once, before turning it on.

  1. 1
    Connect Brave SearchWeb, news, image, video search.
  2. 2
    Connect OpenAIModels, embeddings, files.
  3. 3
    Connect BigQueryDatasets, queries, schemas.
  4. 4
    Set each agent's modelWe leave models unset so you pick the tier — fast + cheap, or top-quality.
  5. 5
    Tune it to your dataEdit the prompts, filters, and field mappings so it matches how your team works.
  6. 6
    Test, then turn it onRun once against a sample, confirm the output, then enable the trigger.

Run this workflow in your colony.

14-day trial. No DevOps. No Sales call. Provisioned in under a minute.