My LLM misses items when I ask it to find every instance of something in a long document. Should I split the prompt or the document?

Usually split: models miss instances in long inputs even when the input fits in the context window. DocETL, our open-source system, rewrites pipelines to decompose the document or the operation; our DocETL paper uses this exact case, finding every force majeure or indemnification clause in long contracts. In MOAR, our later optimizer for DocETL, one example rewrite turns one compound filter into two simpler filters.

Sources: DocETL: Agentic Query Rewriting and Evaluation for Complex Document Processing (VLDB 2025); Multi-Objective Agentic Rewrites for Unstructured Data Processing (VLDB 2026)

Guide: How to Process Documents with LLMs Cheaply

More questions on this topic

All questions about using AI for data and knowledge work