Interfaces for data work with LLMs
LLMs now run over whole datasets: they classify documents, extract fields, summarize transcripts, and answer questions over retrieved text. The model is only part of the work. People still decide what to ask, check whether the answers are right, and read the results. We study how people do that work and build interfaces that make it faster.
Steering an LLM pipeline over many documents
A prompt that works on one example often fails on the hundredth document. DocWrangler (UIST 2025 Best Paper Honorable Mention), with Bhavya Chopra, Mawil Hasan, Stephen Lee, Björn Hartmann, Joe Hellerstein, Aditya Parameswaran, and Eugene Wu, is an IDE for DocETL pipelines. It lets you annotate documents and outputs in place; an LLM turns your notes into revised prompts, or suggests splitting an operation that is too hard. In our study and public deployment, users often turned open-ended operations into classifiers so they were easier to check.
Debugging retrieval-augmented generation
When a RAG answer is wrong, the fault can be in retrieval or in generation. RAGGY (CHI 2026 Best Paper), led by Quentin Romero Lauro, with Sepanta Zeighami and Aditya Parameswaran, pairs composable RAG building blocks with an interface for fast, interactive debugging, based on a study of how 12 engineers debug RAG pipelines.
Finding a dataset
Keyword search sees only titles and tags, so you often can’t tell whether a dataset fits your task until you inspect its rows. DataScout (UIST 2025), led by Rachel Lin, with Bhavya Chopra, Wenjing Lin, Madelon Hulsebos, and Aditya Parameswaran, suggests query rewrites, searches dataset contents, and shows how relevant each dataset is to your task.
Reading AI-generated text at scale
A column of AI summaries is hard to skim when every row uses different words and structure. Five Principles for Generating Readable AI Text in Bulk draws on reading research for rules such as starting each item with its subject or verb.
Questions
- How do I make AI-generated summaries in a dashboard or table easy to skim?
- How do I get an LLM to write consistent one-line titles or summaries for many items?
- Is there a better way to iterate on prompts when an LLM pipeline runs over many documents?
- Why is it so hard to find a good dataset for my project?
- My RAG answers are wrong. How do I tell whether retrieval or generation is the problem?
- How do I find which cells in a messy Jupyter notebook actually produced this plot or model, so I can clean it up?
- When I change an earlier Jupyter cell, how do I know which later cells to rerun?