Nightly Citation Faithfulness Auditor for Coda-Grounded Answers

Audits your answer bot for hallucination drift. On a schedule, it pulls the previous day's logged answers and their cited Coda row IDs, then uses OpenAI as a judge to verify that every claim in each answer is genuinely backed by the cited row content. It scores faithfulness and records the verdict so you can catch a degrading retrieval pipeline before users do.

Use it once you have a grounded answer bot in production and need ongoing assurance that answers stay tied to source. Good for compliance-sensitive teams who must prove answers were citation-backed.

1. 1A nightly schedule triggers the audit run.
2. 2Postgres returns yesterday's answer log rows, each with its question, answer text, and cited Coda row IDs.
3. 3The workflow fetches the current content of each cited row from Coda.
4. 4OpenAI judges, claim by claim, whether the answer is fully supported by the cited rows and returns a faithfulness score plus a list of unsupported claims.
5. 5A logic step routes any answer below the pass threshold to a Slack alert; all scores are written back to Postgres for trend tracking.