What is the fastest, cheapest way to run an LLM over every row of a large dataset?

Use Quail, our open-source engine for AI functions in SQL: it runs an open model as an offline batch job, keeps the GPU full, and plans the query and the inference together to reuse work across rows. Two AI filters over all 100,000 IMDB reviews cost $0.37 on one H100 at Modal's price, including model startup. The same workload at GPT-5 nano prices would cost about $1.75.

Sources: Building an Ultra-High Throughput AI-SQL Engine

Guide: How to Process Documents with LLMs Cheaply

Guide: What Is AI-SQL?

More questions on this topic

All questions about using AI for data and knowledge work