Zero-hallucination RAG: grounded answers over 936 Lex Fridman episodes

A chatbot that answers questions about podcast episodes has one unforgivable failure mode: inventing a quote. "What did Lex say about loneliness?" — and the model confidently produces a sentence that sounds exactly like Lex but was never spoken.

OmniPod is a chat over 936 Lex Fridman episodes where every answer is grounded, verified against the actual transcript before it reaches the user. If the source doesn't support the answer, it says so. The whole system is 1,138 lines of Python across 9 files.

Corpus and ingestion: no API keys

Two scrapers feed the index, both free: official transcript pages on lexfridman.com (requests + BeautifulSoup) and YouTube auto-captions via a public proxy. No YouTube API key, no paid source. All 936 episodes ingest in about 8 minutes.

Chunking is deliberately boring: 512 characters with 128 overlap, each chunk tagged with its episode and guest metadata so retrieval can filter by guest. That yields 19,140 chunks, small enough that a single laptop instance handles the whole corpus.

Retrieval: local embeddings, one vector DB

Chunks are embedded with bge-small-en-v1.5 (384 dims) running locally on MPS, ~100ms per query embedding. Search is cosine similarity in Qdrant: ~50ms at 19K points, with guest metadata filtering built in.

Why this stack, concretely:

• bge-small over a bigger model: 384-dim vectors are fast to search and good enough for conversational podcast text. It runs on a laptop GPU, so embedding costs $0 and never leaves the machine.
• Qdrant over an embedded store: filterable metadata out of the box, and cosine search at this scale is milliseconds.

The router: one prompt fails at scale

Not every question wants the same pipeline. "What did Huberman say about sleep?" and "Compare AI safety views across guests" and "Write an essay on consciousness" are three different problems. OmniPod classifies intent first, then dispatches:

An LRU cache avoids re-embedding repeated queries, and a semaphore caps concurrent LLM calls at 5 so bursty sessions don't blow the budget.

The groundedness check

Every generated answer runs through verify_groundedness() before it's shown:

# verify_groundedness() — the last gate before the UI
def verify_groundedness(answer: str, context: list[Chunk]) -> str:
    prompt = (
        "Check every factual claim in the answer against the context. "
        "For each claim return SUPPORTED, UNSUPPORTED, or NOT_IN_CONTEXT. "
        "If any claim is unsupported or missing, name it and explain why."
    )
    verdict = llm(prompt, answer=answer, context=context)
    if verdict.strip() != "OK":
        return refuse_or_rephrase(answer, verdict)  # says "not covered" — never guesses
    return answer

The model must defend each claim against the retrieved text. When the transcripts don't cover a question, the answer is explicit about it instead of papering over the gap.

Numbers that matter

Lessons

• The verify step enforces groundedness. That's what makes "no hallucinations" a property of the pipeline instead of a hope.
• Route by intent before doing anything else. One retrieve-answer prompt for every question type is what makes RAG chatbots feel dumb.
• Small and local wins at this scale. A 384-dim model on a laptop GPU, one vector DB container, one LLM call, and the whole thing answers in ~2 seconds.

The system answers with confidence only where the transcripts are confident, and every answer carries a source you can check.