ENGINEERING

Nightly README code example drift detector

On a nightly schedule, extracts every fenced code block from your README files, runs each one against the latest installed SDK.

CategoryEngineering
Enginesim
Difficultyintermediate
Triggerschedule
Steps6
Setup~15 min

How it runs

The automated pipeline, trigger to output.

  • TriggerNightly schedule fires
  • ActionClone repo and install latest published packageShell
  • ActionExtract fenced code blocks from markdown into scriptsShell
  • ActionExecute each snippet in a sandbox, capture exit code and stderrShell
  • LogicSplit passing vs failing; format failures with file and line
  • OutputPost drift digest to SlackSlack

What it does

Documented examples rot silently. This workflow runs every night, pulls every fenced code block out of your README and docs markdown, executes each one in an isolated shell against the currently published version of your library, and reports which examples have broken. You get a single Slack digest listing the exact file, line, and error for every drifted snippet.

When to use it

Use it when your README contains copy-pasteable code samples that must keep working as the API evolves. Ideal for SDKs, CLIs, and libraries where a stale example damages developer trust on first contact.

How it works

  1. 1A nightly schedule fires the run.
  2. 2A shell step clones the repo and installs the latest published package version.
  3. 3A shell step parses all markdown files, extracting fenced blocks tagged as runnable code into individual scripts.
  4. 4Each extracted snippet is executed in a sandboxed shell; exit codes and stderr are captured.
  5. 5A logic step splits passing from failing snippets and formats the failures with file path, line number, and error text.
  6. 6A Slack message delivers the digest, or a quiet all-clear when nothing drifted.

Set it up

What you configure once, before turning it on.

  1. 1
    Connect ShellRun sandboxed commands inside the workspace.
  2. 2
    Connect SlackChannels, DMs, threads, mentions.
  3. 3
    Set each agent's modelWe leave models unset so you pick the tier — fast + cheap, or top-quality.
  4. 4
    Tune it to your dataEdit the prompts, filters, and field mappings so it matches how your team works.
  5. 5
    Test, then turn it onRun once against a sample, confirm the output, then enable the trigger.

Run this workflow in your colony.

14-day trial. No DevOps. No Sales call. Provisioned in under a minute.