My RAG answers are wrong. How do I tell whether retrieval or generation is the problem?
We recommend testing the pieces separately; for example, give the model known-good context and see whether the answer improves. In RAG Without the Lag (CHI 2026 Best Paper), we studied how 12 engineers debug RAG pipelines and built RAGGY, which pairs composable RAG building blocks with an interface for fast, interactive debugging. We found the settings that matter most often need hours of preprocessing after each change, which slows debugging.
Sources: RAG Without the Lag: Interactive Debugging for Retrieval-Augmented Generation Pipelines (CHI 2026)
Guide: Interfaces for Data Work with LLMs
More questions on this topic
- How do I make AI-generated summaries in a dashboard or table easy to skim?
- How do I get an LLM to write consistent one-line titles or summaries for many items?
- Is there a better way to iterate on prompts when an LLM pipeline runs over many documents?
- Why is it so hard to find a good dataset for my project?
- How do I find which cells in a messy Jupyter notebook actually produced this plot or model, so I can clean it up?
- When I change an earlier Jupyter cell, how do I know which later cells to rerun?