ENGINEERING

Grade dependency-bump PRs by changelog breaking-change signals

When a dependency-bump PR opens, fetches the package's changelog for the version range, uses an LLM to score breaking-change risk, labels the PR low/medium/high.

CategoryEngineering
Enginesim
Difficultyintermediate
Triggerevent
Steps6
Setup~15 min

How it runs

The automated pipeline, trigger to output.

  • TriggerPR opened touching a dependency manifestGitHubGitHub
  • ActionFetch changelog for old→new version rangeGitHubGitHub
  • ActionLLM grades breaking-change risk + rationaleOpenAI
  • LogicMap verdict to low/medium/high
  • ActionApply risk label to the PRGitHubGitHub
  • OutputPost grade and rationale to SlackSlack

What it does

Every time a bot or contributor opens a dependency-bump pull request, this workflow pulls the changelog entries between the old and new version, asks an LLM to flag breaking-change signals (removed APIs, behavior changes, peer-dep bumps, major version jumps), assigns a risk grade, applies a GitHub label, and notifies the team.

When to use it

Use it when Dependabot or Renovate volume is too high to review by hand and you want a fast triage signal before anyone spends time on a PR. It separates the rubber-stamp `low` bumps from the `high` ones that need a human and a test run.

How it works

  1. 1A PR-opened webhook fires on PRs that touch a manifest file (package.json, go.mod, etc.).
  2. 2The flow parses the package name plus old and new versions from the PR title and diff.
  3. 3It fetches the changelog/release notes for that version range from GitHub.
  4. 4An LLM classifies the notes into low/medium/high risk with a one-line rationale.
  5. 5The matching `risk:low|medium|high` label is applied to the PR.
  6. 6A Slack message summarizes the grade, rationale, and PR link for the eng channel.

Set it up

What you configure once, before turning it on.

  1. 1
    Connect GitHubRepos, issues, pull requests, actions.
  2. 2
    Connect OpenAIModels, embeddings, files.
  3. 3
    Connect SlackChannels, DMs, threads, mentions.
  4. 4
    Set each agent's modelWe leave models unset so you pick the tier — fast + cheap, or top-quality.
  5. 5
    Tune it to your dataEdit the prompts, filters, and field mappings so it matches how your team works.
  6. 6
    Test, then turn it onRun once against a sample, confirm the output, then enable the trigger.

Run this workflow in your colony.

14-day trial. No DevOps. No Sales call. Provisioned in under a minute.