AI & RAG

Runbook Drift Detector for Stale Remediation Steps

Runs on a schedule to compare each Confluence runbook against the real systems it references.

CategoryAI & RAG
Enginepaperclip
Difficultyadvanced
Triggerschedule
Steps5
Setup~25 min

How it runs

The automated pipeline, trigger to output.

  • TriggerScheduled weekly runbook audit fires
  • ActionList and read runbooks from ConfluenceConfluenceConfluence
  • ActionCross-check referenced Datadog monitors/dashboardsDatadogDatadog
  • LogicIsolate steps with missing or renamed references
  • OutputFile Linear ticket per stale step with locationLinearLinear

What it does

Keeps runbooks honest. On a schedule, the agent reads each Confluence runbook, extracts the concrete resources it tells operators to touch — dashboards, services, commands — and checks them against live signals from Datadog. When a step references something that no longer exists or has been renamed, it files a Linear ticket flagging the stale step and its location.

When to use it

Use it when responders lose trust in runbooks because steps reference dead dashboards or retired services. Best for ops teams that want documentation maintained proactively instead of discovered broken mid-incident.

How it works

  1. 1A scheduled trigger fires the audit (e.g. weekly).
  2. 2The agent lists and reads runbooks from Confluence, capturing each step and version.
  3. 3It cross-checks referenced Datadog monitors and dashboards for existence.
  4. 4A logic step isolates steps whose referenced resources are missing or renamed.
  5. 5For each drift, it opens a Linear ticket naming the runbook, step number, and the broken reference.

Set it up

What you configure once, before turning it on.

  1. 1
    Connect ConfluenceSpaces, pages, blueprints.
  2. 2
    Connect DatadogMetrics, traces, log search.
  3. 3
    Connect LinearIssues, projects, cycles, triage.
  4. 4
    Set each agent's modelWe leave models unset so you pick the tier — fast + cheap, or top-quality.
  5. 5
    Tune it to your dataEdit the prompts, filters, and field mappings so it matches how your team works.
  6. 6
    Test, then turn it onRun once against a sample, confirm the output, then enable the trigger.

Run this workflow in your colony.

14-day trial. No DevOps. No Sales call. Provisioned in under a minute.