AI & RAG

Incremental Runbook Corpus Indexer for Confluence and Drive

Runs on a schedule to detect changed Confluence pages and Drive docs, chunk and re-embed only what moved, and keep the runbook vector index in Postgres fresh.

CategoryAI & RAG
Enginesim
Difficultyadvanced
Triggerschedule
Steps6
Setup~25 min

How it runs

The automated pipeline, trigger to output.

  • TriggerScheduled run (hourly or nightly)
  • ActionList Confluence pages changed since watermarkConfluenceConfluence
  • ActionList Drive files modified since watermarkGoogle DriveGoogle Drive
  • LogicClassify items as new, updated, or deleted
  • ActionChunk and embed changed contentOpenAI
  • OutputUpsert vectors and prune stale chunks in PostgresPostgreSQLPostgres

What it does

Keeps the knowledge base behind your runbook agent current without re-processing everything. On each run it lists Confluence pages and Drive files modified since the last watermark, fetches their content, splits it into overlapping chunks, generates embeddings, and upserts them into the Postgres index, deleting chunks for removed or archived sources.

When to use it

Use it as the ingestion backbone for any of the answer agents in this theme. Run it nightly or hourly so citations always point at the latest version of a procedure rather than a stale snapshot.

How it works

  1. 1A scheduled trigger fires on your chosen cadence.
  2. 2Confluence is queried for pages changed since the stored watermark; Drive is queried in parallel for recently modified files.
  3. 3A logic step diffs results against the index to classify each item as new, updated, or deleted.
  4. 4New and updated content is chunked and embedded via OpenAI.
  5. 5The vectors and source metadata are upserted into Postgres, stale chunks are pruned, and the watermark is advanced.

Set it up

What you configure once, before turning it on.

  1. 1
    Connect ConfluenceSpaces, pages, blueprints.
  2. 2
    Connect Google DriveDocs, sheets, slides, files.
  3. 3
    Connect OpenAIModels, embeddings, files.
  4. 4
    Connect PostgresAny Postgres URL — query, write, migrate.
  5. 5
    Set each agent's modelWe leave models unset so you pick the tier — fast + cheap, or top-quality.
  6. 6
    Tune it to your dataEdit the prompts, filters, and field mappings so it matches how your team works.
  7. 7
    Test, then turn it onRun once against a sample, confirm the output, then enable the trigger.

Run this workflow in your colony.

14-day trial. No DevOps. No Sales call. Provisioned in under a minute.