V-RAG — Voice-Enabled RAG Pipeline
Overview
A low-latency question-answering system for your own documents. Ask by typing or speaking, in English or Hinglish (Hindi + English), and get an answer drawn only from the documents you supplied — with the sources it used.
Everything runs on your machine. No cloud APIs, no vendor keys, no data leaving the laptop.
How a question is answered
Each stage can stop the question. If the documents don’t contain a trustworthy answer, the system refuses rather than inventing one.
- Understand the question. Speech is transcribed; follow-ups like “how do I scale it?” are rewritten into standalone questions, and non-English questions are translated for searching.
- Search twice. Keyword search (BM25) and meaning-based search run in parallel and are merged — one catches exact terms and error codes, the other paraphrases.
- Re-read the shortlist. A cross-encoder re-reads the top candidates against the question and reorders them.
- Decide whether to answer. Below a 0.65 confidence score the pipeline stops and says it doesn’t know. The language model is never called.
- Answer under constraint. Only the supplied passages, temperature 0.0.
- Check the answer. Names and facts are verified against the source text with spaCy NER, and the sources used are attached.
How well it works
Measured against 67 questions — 47 answerable from the documents, 20 designed to tempt the system into making something up.
| Measure | Target | Result |
|---|---|---|
| Answers supported by the documents | ≥ 85% | 87% |
| Answers that address the question | ≥ 85% | 95% |
| Trick questions correctly refused | 100% | 20 of 20 |
| Answerable questions answered | ≥ 90% | 94% |
How fast it is
On a laptop CPU, no GPU:
| Time | |
|---|---|
| Typed question, first time | ~2.3 s |
| Typed question, asked again | ~2 ms |
| Question it refuses | ~0.7 s |
| Spoken question to spoken answer | ~9 s |
Hinglish
Hindi and English mixed together, the way people actually speak. A dedicated speech model transcribes it, the question is translated to English for searching — the documents are in English — and the answer is read back aloud in the English voice.
Credits
Built on the S-Y-N-C-H-R-O-N-I-C framework described by Vishal Mysore in How to Design a RAG Pipeline for 10 Million Documents with Zero Hallucination. The framework supplied the ten design principles; this is an independent implementation of them on a fully local stack.