AI & RAG

Weekly runbook citation freshness auditor

Scans every runbook page the answer bot cites, flags pages not updated since their referenced code last changed in GitHub.

CategoryAI & RAG
Enginesim
Difficultyintermediate
Triggerschedule
Steps6
Setup~15 min

How it runs

The automated pipeline, trigger to output.

  • TriggerWeekly schedule fires the freshness audit
  • ActionList runbook pages and edit dates from ConfluenceConfluenceConfluence
  • ActionFetch latest commit date for each cited repo pathGitHubGitHub
  • LogicFlag pages where code changed after the doc's last edit
  • ActionRecord staleness scores in Postgres audit tablePostgreSQLPostgres
  • OutputPost worst-offenders digest to docs Slack channelSlack

What it does

Keeps the knowledge base honest by detecting runbook pages whose cited procedures may be out of date. It cross-references each Confluence page's last-edited date against the most recent change to the GitHub paths it documents, surfacing drift before it misleads an engineer.

When to use it

Run it weekly when your runbook wiki is large enough that no one manually reviews freshness, and you need confidence that cited answers reflect the current system rather than last quarter's architecture.

How it works

  1. 1A weekly schedule kicks off the audit.
  2. 2The flow pulls the list of runbook pages and their last-modified timestamps from Confluence.
  3. 3For each page, it reads the referenced repo paths and queries GitHub for the latest commit touching them.
  4. 4A comparison flags any page whose code moved after the doc's last edit, scoring severity by how far behind it is.
  5. 5Flagged pages are written to a Postgres audit table for trend tracking.
  6. 6A digest of the worst offenders posts to the docs channel in Slack.

Set it up

What you configure once, before turning it on.

  1. 1
    Connect ConfluenceSpaces, pages, blueprints.
  2. 2
    Connect GitHubRepos, issues, pull requests, actions.
  3. 3
    Connect PostgresAny Postgres URL — query, write, migrate.
  4. 4
    Connect SlackChannels, DMs, threads, mentions.
  5. 5
    Set each agent's modelWe leave models unset so you pick the tier — fast + cheap, or top-quality.
  6. 6
    Tune it to your dataEdit the prompts, filters, and field mappings so it matches how your team works.
  7. 7
    Test, then turn it onRun once against a sample, confirm the output, then enable the trigger.

Run this workflow in your colony.

14-day trial. No DevOps. No Sales call. Provisioned in under a minute.