DEVOPS

Post-Deploy Smoke-Check Summary

After a Vercel production deploy, runs Datadog Synthetic smoke checks against the new build and posts a pass/fail summary to Slack.

CategoryDevOps
Enginesim
Difficultyintermediate
Triggerwebhook
Steps5
Setup~15 min

How it runs

The automated pipeline, trigger to output.

  • TriggerVercel deployment succeededVercelVercel
  • ActionTrigger Datadog Synthetic smoke tests against deploy URLDatadogDatadog
  • LogicPoll results and aggregate pass/fail countsDatadogDatadog
  • LogicBranch on overall health (all passed vs. any failed)
  • OutputPost smoke-check summary to Slack channelSlack

What it does

Watches for successful production deployments on Vercel and immediately runs a smoke-check pass against the live build. It triggers your Datadog Synthetic tests for the critical paths (homepage, login, checkout, key API endpoints), waits for the results, and then posts a single, readable summary to Slack: which checks passed, which failed, response times, and a direct link back to the deployment. No more wondering whether the deploy that just went out is actually serving real traffic correctly.

When to use it

Use this when you ship to production frequently and want fast confirmation that a deploy is healthy without anyone manually clicking through the app. It is ideal for teams practicing continuous deployment who need a tight feedback loop: the moment Vercel reports a deploy as ready, the workflow confirms the site is actually up and the golden paths work. Pair it with an on-call rotation so a failed smoke check turns into an immediate, actionable Slack alert instead of a customer complaint hours later.

How it works

A Vercel deployment-succeeded webhook starts the run and hands over the deployment URL and metadata. The workflow calls the Datadog API to trigger your designated Synthetic smoke tests against that URL, then polls until all results are in. It aggregates the outcomes — counting passes and failures, capturing latency and any error detail — and branches on the result. Either way it formats a compact summary (status emoji, per-check breakdown, deploy link) and sends it to your chosen Slack channel, flagging the message clearly when any check fails so the right people see it first.

Set it up

What you configure once, before turning it on.

  1. 1
    Connect VercelDeploys, runtime logs, analytics.
  2. 2
    Connect DatadogMetrics, traces, log search.
  3. 3
    Connect SlackChannels, DMs, threads, mentions.
  4. 4
    Set each agent's modelWe leave models unset so you pick the tier — fast + cheap, or top-quality.
  5. 5
    Tune it to your dataEdit the prompts, filters, and field mappings so it matches how your team works.
  6. 6
    Test, then turn it onRun once against a sample, confirm the output, then enable the trigger.

Run this workflow in your colony.

14-day trial. No DevOps. No Sales call. Provisioned in under a minute.