MARKET RESEARCH
Forum and community review mining into a BigQuery theme warehouse
On a nightly schedule, crawls Reddit and product-forum threads, embeds and clusters the posts with a HuggingFace model.
How it runs
The automated pipeline, trigger to output.
- TriggerNightly schedule starts the crawl
- ActionCrawl forum and subreddit postsFirecrawl
- ActionEmbed and assign posts to themesHugging Face
- LogicLabel theme, sentiment, and permalink
- ActionStream labeled rows into BigQueryBigQuery
- OutputSend new/growing-theme summary to SlackSlack
What it does
Turns messy community discussion into a structured, queryable dataset. Each night it gathers forum and subreddit posts about your product, clusters them into themes, and lands one row per post in BigQuery tagged with its theme, sentiment, and source so you can chart demand trends and slice by release.
When to use it
Use it when a one-off digest isn't enough and you need a durable history of what the community discusses — to correlate theme volume against launches, or feed a dashboard the whole org trusts.
How it works
- 1A nightly schedule starts the crawl.
- 2Firecrawl pulls posts and comments from the configured forum URLs and subreddits.
- 3A HuggingFace model embeds each post, assigns it to the nearest existing theme, or opens a new one when nothing fits.
- 4Logic labels each post with theme, coarse sentiment, and a permalink for traceability.
- 5Rows are streamed into a BigQuery table partitioned by date for trend analysis.
- 6A short run summary of new and fastest-growing themes is sent to Slack.
Set it up
What you configure once, before turning it on.
- 1Connect FirecrawlCrawl, scrape, structured extract.
- 2Connect Hugging FaceModels, datasets, spaces — the open-source hub.
- 3Connect BigQueryDatasets, queries, schemas.
- 4Connect SlackChannels, DMs, threads, mentions.
- 5Set each agent's modelWe leave models unset so you pick the tier — fast + cheap, or top-quality.
- 6Tune it to your dataEdit the prompts, filters, and field mappings so it matches how your team works.
- 7Test, then turn it onRun once against a sample, confirm the output, then enable the trigger.
More Market Research workflows
Enrich Inbound Accounts with BigQuery Firmographics and Score Fit
When a new account row lands in Airtable, joins it against BigQuery public business datasets to attach firmographic attributes.
Blend BigQuery TAM with Live Competitor Signals into a Notion Brief
On demand, sizes a chosen segment from BigQuery public data, gathers current competitor signals via Brave Search, and synthesizes a one-page market brief into Notion.
Allocate Sales Territory TAM from BigQuery Geo Data to HubSpot
When triggered by a webhook, queries BigQuery public ZIP-level business data to compute TAM per sales territory.
Hiring Surge Detector with Slack Alert
Detects when a target account's open-role count jumps above its recent baseline and posts a ranked Slack alert to the GTM channel so reps can act on a company that is clearly…
Tech-Stack Shift Inference from Job Descriptions
Reads new job descriptions for target accounts, uses an LLM to extract named technologies and infer stack changes.
Weekly Hiring-Intel Briefing for GTM
An agent reviews the week's accumulated hiring signals across all target accounts, writes a narrative briefing that infers each account's likely initiatives.
Run it inside a business
This workflow drops into a full company template. Import the org, and this is one of the playbooks its agents run.
Run this workflow in your colony.
14-day trial. No DevOps. No Sales call. Provisioned in under a minute.