DEVOPS

Fail GitLab MR when Vercel preview regresses Lighthouse vs production baseline

Compares the Vercel preview's Lighthouse scores against the current production deployment's scores and blocks the GitLab MR only when a category drops more than the allowed delta.

CategoryDevOps
Enginesim
Difficultyadvanced
Triggerwebhook
Steps6
Setup~25 min

How it runs

The automated pipeline, trigger to output.

  • TriggerVercel preview deployment readyVercelVercel
  • ActionLighthouse on preview URLShell
  • ActionLighthouse on production URL (baseline)Shell
  • LogicCompute per-category delta vs tolerance
  • ActionFail GitLab MR on regressionGitLabGitLab
  • OutputComment baseline-vs-preview diff on MRGitLabGitLab

What it does

Instead of fixed thresholds, this gate measures regression: it scores the preview, scores the live production URL as a baseline, and fails the MR only when a category falls by more than the configured tolerance. This avoids blocking branches that were already below an absolute floor before the change.

When to use it

Use it on mature apps where absolute thresholds are unrealistic but silent regressions are unacceptable. It answers "did this change make things worse?" rather than "is this perfect?"

How it works

  1. 1A Vercel deployment-ready webhook provides the preview URL and MR reference.
  2. 2A shell step runs Lighthouse against the preview URL.
  3. 3A second shell step runs Lighthouse against the production URL to establish the baseline.
  4. 4A logic branch computes per-category deltas and flags any drop beyond tolerance.
  5. 5If a regression is flagged, the GitLab MR pipeline is set to failed.
  6. 6A comment posts the baseline-vs-preview table with the deltas highlighted.

Set it up

What you configure once, before turning it on.

  1. 1
    Connect VercelDeploys, runtime logs, analytics.
  2. 2
    Connect ShellRun sandboxed commands inside the workspace.
  3. 3
    Connect GitLabRepos, MRs, pipelines, registry.
  4. 4
    Set each agent's modelWe leave models unset so you pick the tier — fast + cheap, or top-quality.
  5. 5
    Tune it to your dataEdit the prompts, filters, and field mappings so it matches how your team works.
  6. 6
    Test, then turn it onRun once against a sample, confirm the output, then enable the trigger.

Run this workflow in your colony.

14-day trial. No DevOps. No Sales call. Provisioned in under a minute.