Can I use a cheaper LLM for most rows without losing accuracy?

Use BARGAIN, our method for LLM-powered data processing with guarantees: a cheap model handles a row when it can be trusted, and BARGAIN decides when that is with adaptive sampling and statistical estimates, with guarantees on accuracy, precision, or recall. Across 8 datasets, BARGAIN reduced cost by up to 86% more than prior methods.

How it works. A model cascade sends each row to a cheap model first and keeps its answer only when the model's confidence is above a threshold; the other rows go to the expensive model. The hard part is the threshold: set it too low and accuracy drops, too high and you save little. BARGAIN checks the cheap model on an adaptively chosen sample of rows and uses statistical estimates to pick the threshold that saves the most while still meeting the accuracy, precision, or recall target you set.

Sources: Cut Costs, Not Accuracy: LLM-Powered Data Processing with Guarantees (SIGMOD 2026)

Guide: How to Process Documents with LLMs Cheaply

More questions on this topic

All questions about using AI for data and knowledge work