Independent project
Multimodal RAG Knowledge Assistant with Voice Interface
A Retrieval-Augmented Generation assistant that answers technical-support questions from a document corpus, cites the passages it used, and says so when it cannot answer.
The problem
Language models answer fluently even when they are wrong. For technical support, an answer is only useful if it is grounded in the actual documentation and the reader can check where it came from.
What I built
- An ingestion pipeline for PDF, DOCX, TXT, Markdown and HTML documents.
- Dense retrieval over a FAISS index with cross-encoder reranking on top.
- Grounded generation: every answer returns a numbered passage-citation panel.
- Explicit abstention when the retrieved context cannot support an answer.
- A voice layer with speech-to-text capture and spoken answers for hands-free use.
- A FastAPI inference backend with a Streamlit operator interface.
My engineering contribution
- Designed and built the pipeline end to end, from document parsing to the cited answer.
- Set up evaluation with RAGAS against a 50-question golden set.
- Added embedding caching and asynchronous retrieval to the serving path.
- Containerised the system with Docker and tested it in a GitHub Actions CI pipeline.
Architecture and workflow
PDF, DOCX, TXT, Markdown and HTML files are loaded into the corpus.
Technical decisions
- Cross-encoder reranking over dense retrieval, trading a little latency for better passages in the prompt.
- Abstain rather than guess: an explicit “cannot answer from the corpus” beats a confident hallucination.
- Measure retrieval and faithfulness with RAGAS on a fixed golden set instead of judging answers by eye.
Outcomes
- Every answer is traceable to numbered source passages.
- The system abstains explicitly when the retrieved context cannot support an answer.
- Evaluated with RAGAS against a 50-question golden set.