AI & RAG

Low-Confidence RFC Question Escalation to Linear

When the answer-bot cannot find a confident grounded answer to an engineering question, it opens a Linear issue routed to the owning team so the knowledge gap gets documented…

CategoryAI & RAG
Enginesim
Difficultyintermediate
Triggerwebhook
Steps5
Setup~15 min

How it runs

The automated pipeline, trigger to output.

  • TriggerQuestion received via webhookHTTP webhook
  • ActionRetrieve top candidate chunksSupabaseSupabase
  • ActionScore grounding confidence + infer ownerOpenAI
  • LogicBranch on confidence threshold
  • OutputOpen Linear issue for the owning teamLinearLinear

What it does

Closes the loop on questions the RAG bot can't answer. A question arrives via webhook; the flow retrieves candidate sources and scores confidence. If retrieval is weak or the model declines to answer from sources, instead of a dead-end reply it files a Linear issue tagged to the team that owns the relevant area, capturing the question and the closest near-miss sources.

When to use it

When you want unanswered questions to become tracked documentation work rather than vanishing. Pairs with the Slack answer-bot to handle its refusal path.

How it works

  1. 1An incoming question hits an HTTP webhook from the bot's refusal path.
  2. 2The flow retrieves the top candidate chunks from the Supabase vector store.
  3. 3OpenAI scores grounding confidence and identifies the likely owning area.
  4. 4A logic gate routes high-confidence cases back as an answer and low-confidence cases onward.
  5. 5For low confidence, the flow creates a Linear issue with the question, near-miss citations, and the inferred owning team as assignee.

Set it up

What you configure once, before turning it on.

  1. 1
    Connect HTTP webhookTrigger any URL on agent actions.
  2. 2
    Connect SupabaseTables, auth, storage, edge functions.
  3. 3
    Connect OpenAIModels, embeddings, files.
  4. 4
    Connect LinearIssues, projects, cycles, triage.
  5. 5
    Set each agent's modelWe leave models unset so you pick the tier — fast + cheap, or top-quality.
  6. 6
    Tune it to your dataEdit the prompts, filters, and field mappings so it matches how your team works.
  7. 7
    Test, then turn it onRun once against a sample, confirm the output, then enable the trigger.

Run this workflow in your colony.

14-day trial. No DevOps. No Sales call. Provisioned in under a minute.