In what order should I run several LLM filters on the same data?
Run first the filters that reject the most rows for the least cost, so later filters see fewer rows. In How to Cost Your AI-Powered Filters, we rank each later filter by its cost per row divided by the fraction it rejects, following Hellerstein and Stonebraker, and also try each filter first. For three filters over 5,000 IMDB reviews, the chosen order scored 6.87 s compared with 7.30 s for the worst candidate.
Sources: How to Cost Your AI-Powered Filters
Guide: How to Process Documents with LLMs Cheaply
Guide: What Is AI-SQL?
More questions on this topic
- Why are AI functions in SQL (Snowflake Cortex, BigQuery AI, Databricks ai_query) so slow and expensive on big tables?
- How do I extract information accurately from thousands of long documents with an LLM?
- My LLM misses items when I ask it to find every instance of something in a long document. Should I split the prompt or the document?
- Should I ask an LLM to extract dozens of fields from a long document in one prompt, or split it up?
- How do I optimize a multi-step LLM pipeline for both accuracy and cost, not just one prompt at a time?
- Can I use a cheaper LLM for most rows without losing accuracy?
- Is there something better than a model cascade for LLM classification over documents?
- How do I match records across two tables with an LLM without paying for every pair?
- Why is my LLM classifier inaccurate when I label records one at a time, even with a good prompt?
- Should I put many records in one prompt so the LLM sees context when classifying?
- How do I get consistent 1-10 scores from an LLM across many items?
- How do I run SQL queries over thousands of similar PDFs, such as reports or forms that follow one template?
- Why does RAG fail on "how many" or aggregate questions across a whole document collection?
- How do I deduplicate records with an LLM without missing too many true matches?
- How do I add a summary of each user's history and feedback to my prompts without slowing down responses?
- Can LLM agents reliably answer business questions over real company databases, not just text-to-SQL benchmarks?
- Why do text-to-SQL and data agents return wrong answers? Is it picking the wrong table?
- Should an agent use regex or an LLM to pull values like dates out of free-text columns?
- Do AI agents need a different kind of database than human analysts do?
- Why does my LLM agent run so many SQL queries for one question, and how do I cut them down?
- How do I write assertions or guardrails that catch bad LLM outputs in production?
- Can an LLM generate the test criteria or assertions for my prompt?
- How do I catch bad data before it gets into a retraining run without drowning in false alerts?
- How do I monitor an ML model in production when labels arrive late or not at all?
- My model's accuracy on a rolling window dropped. Is the model actually getting worse?
- Can I use Prometheus and Grafana to monitor ML model accuracy?
- Should I fine-tune a model or use RAG to give an LLM my company's knowledge?
- Do long-context models make RAG obsolete?
- How should I evaluate a RAG system in production?