What is AI-SQL?

AI-SQL, also called AI functions, extends SQL with functions that call an LLM. You write each function as a natural-language prompt. For example, this query keeps the reviews that discuss the ending:

SELECT *
FROM reviews AS r
WHERE AI.IF(PROMPT('Does this review discuss the ending?\n\n{0}', r.review));

Snowflake, BigQuery, Databricks, and MotherDuck all support AI-SQL.

Running it is expensive. An AI function evaluates its prompt row by row, so one query can make hundreds of thousands or millions of model calls. A filter needs one LLM call per row. A naive join needs one LLM call for every pair of rows in its two input tables.

Our lab is building Quail, an open-source inference engine for AI functions in SQL. Quail plans the query and the LLM inference together, on open-weight models. Across 29 QUAIL-B queries, it is 1.84x faster on average than well-tuned vLLM baselines.

Posts

Copy That? Faster AI-Powered Extraction by Drafting from the Document
How Quail speeds up LLM extraction with drafts copied from the document, a draft length chosen for each request, and up to 19x speedups over naive vLLM.
To Classify, Try a Trie Tree
How should an AI-SQL engine classify documents with LLMs at scale? We walk through the ways Quail can do it, and how it picks the cheapest one for a given LLM.
How to Cost Your AI-Powered Filters
How fast could your AI-powered filters run? We do the math for an H100 and show how filter order changes the answer.
Building an Ultra-High Throughput AI-SQL Engine
Quail jointly plans AI-SQL queries and model inference. Across 29 QUAIL-B queries, it is 1.84x faster on average than well-tuned vLLM baselines.

For common questions about AI-SQL and LLM inference, see our questions page.

Code