MARKET RESEARCH
Daily Competitor Pricing Snapshot to BigQuery
Captures structured competitor pricing data every day and appends it to a BigQuery table, building a time-series history for trend analysis and dashboards.
How it runs
The automated pipeline, trigger to output.
- TriggerDaily schedule fires
- ActionScrape competitor pricing pagesFirecrawl
- ActionNormalize pages into pricing rowsOpenAI
- LogicValidate schema and numeric prices
- ActionAppend rows to BigQuery tableBigQuery
- OutputPost load summary to SlackSlack
What it does
Builds a clean, queryable history of competitor prices over time. Every day it scrapes pricing, normalizes it into rows, and appends to BigQuery so analysts can chart trends and run cohort comparisons.
When to use it
Use this when you need pricing as data, not just alerts — for example to model competitor discounting patterns, feed a pricing dashboard, or join against your own win/loss data.
How it works
- 1A daily schedule fires the run.
- 2Firecrawl scrapes each competitor pricing page listed in the config.
- 3OpenAI parses each page into a normalized row set: plan name, monthly price, annual price, seat limits, and capture date.
- 4A logic step validates that prices are numeric and the schema matches before any load.
- 5Valid rows are appended to a partitioned BigQuery pricing table.
- 6A confirmation with row counts and any parse failures is posted to Slack so data quality stays visible.
Set it up
What you configure once, before turning it on.
- 1Connect FirecrawlCrawl, scrape, structured extract.
- 2Connect OpenAIModels, embeddings, files.
- 3Connect BigQueryDatasets, queries, schemas.
- 4Connect SlackChannels, DMs, threads, mentions.
- 5Set each agent's modelWe leave models unset so you pick the tier — fast + cheap, or top-quality.
- 6Tune it to your dataEdit the prompts, filters, and field mappings so it matches how your team works.
- 7Test, then turn it onRun once against a sample, confirm the output, then enable the trigger.
More Market Research workflows
Weekly Materiality-Ranked Filing Digest by Email
Each Friday, aggregates the week's regulatory filings across the coverage universe, ranks them by materiality.
Earnings Keyword-Spike Alert into Linear
Scans new earnings-call transcripts for a set of strategic keywords you care about, and when a competitor's mention frequency spikes above its historical baseline.
Allocate Sales Territory TAM from BigQuery Geo Data to HubSpot
When triggered by a webhook, queries BigQuery public ZIP-level business data to compute TAM per sales territory.
Discover Adjacent Segments from BigQuery and Rank Expansion Bets
On a schedule, scans BigQuery public industry data for fast-growing NAICS sectors adjacent to your core segments, scores them as expansion bets.
Blend BigQuery TAM with Live Competitor Signals into a Notion Brief
On demand, sizes a chosen segment from BigQuery public data, gathers current competitor signals via Brave Search, and synthesizes a one-page market brief into Notion.
Cross-Competitor Theme-Shift Analyst
An agent that gathers the latest earnings transcripts across all tracked competitors, compares this quarter's themes against the prior quarter stored in Coda.
Run it inside a business
This workflow drops into a full company template. Import the org, and this is one of the playbooks its agents run.
Run this workflow in your colony.
14-day trial. No DevOps. No Sales call. Provisioned in under a minute.