Running queries

Every Quail query runs inside a session. The session loads the model, compiles GPU kernels, and tokenizes your documents. Multiple queries on the same session reuse the loaded model and tokenized documents.

Creating a session

import quail

config = quail.EngineConfig(model="qwen3-4b-fp8", device="h100-sxm")
with quail.Session(config=config) as session:
    ...

gpus defaults to 1.

The session is a context manager. Collect result rows before closing it: result collection reads from the session's temporary files. Those files must fit on local disk. Session.close() waits for background tokenization and removes the temporary files.

Registering tables

Register each input table before writing a query. The name you register becomes the SQL table name.

session.register("reviews", quail.DocumentProvider.from_dataset(reviews, id_col="id"))
session.register("aspects", quail.DocumentProvider.from_table(aspects, id_col="id"))

You can register as many tables as you need. These are the available DocumentProvider factories:

Every provider needs an id_col that identifies each document.

Running queries

Once tables are registered, run SQL queries with session.sql(). You can run multiple queries on the same session.

result_1 = session.sql("""
    SELECT r.id FROM reviews r
    WHERE AI.IF(PROMPT('Is this review positive?\n\n{0}', r.body))
""", dialect="bq").collect()

result_2 = session.sql("""
    SELECT r.id, a.aspect FROM reviews r, aspects a
    WHERE AI.IF(PROMPT('Does the review discuss this aspect?\n\n{0}\n\n{1}',
                       r.body, a.aspect))
""", dialect="bq").collect()

The first query loads the model onto the GPU. Later queries on the same session reuse that model, the compiled kernels, and the tokenized documents. sql() returns a Query.

Running on Quail Server

Pass a Quail Server URL as endpoint. sql() then returns a RemoteQuery. Submitted queries continue after the client exits. Another client can retrieve the saved record by its query id.

with quail.Session(config=config, endpoint="http://127.0.0.1:8642") as session:
    ...

Remote queries use SQL. The server compiles and plans them when execution starts. After submission, read the plan from run.status().plan["text"].

Multiple GPUs

Set gpus to use more than one GPU. Each GPU holds one complete model copy.

config = quail.EngineConfig(model="qwen3-4b-fp8", device="h100-sxm", gpus=4)

See Supported models and GPUs for the full list of configurations.