Quail Server

Quail provides an HTTP server for submitting SQL queries and retrieving results. You can connect with the Python Session API.

Start the server

On a machine with a supported GPU, install Quail with the server extra and start it:

uv pip install "quail-engine[server]"
quail-server

The server listens on 127.0.0.1:8642 and uses GPU 0 by default. It saves inputs and results in ~/.quail/server. The first query loads its model. Later queries can reuse it.

Use quail-server --help to see the configuration options.

Connect from another machine

To connect from another machine, set an access token and start the server with --host 0.0.0.0:

export QUAIL_SERVER_TOKEN='replace-with-your-token'
quail-server --host 0.0.0.0

Set the same QUAIL_SERVER_TOKEN on the machine running your Python program.

Use http://<host>:8642 on a trusted private network. On an untrusted network, use an HTTPS reverse proxy to encrypt the token and query data.

Submit a query

Install quail-engine on the machine running your Python program. Pass the server URL as endpoint when you create a session.

For example, the table below has movie review IDs in id and text in body. The query selects reviews that discuss the movie's ending:

import pyarrow as pa
import quail

reviews = pa.table({
    "id": ["r1", "r2"],
    "body": ["The ending was excellent.", "I liked the soundtrack."],
})
config = quail.EngineConfig(model="qwen3-4b-fp8", device="h100-sxm")
endpoint = "http://127.0.0.1:8642"

with quail.Session(config=config, endpoint=endpoint) as session:
    session.register(
        "reviews",
        quail.DocumentProvider.from_table(reviews, id_col="id"),
    )
    query = session.sql("""
        SELECT r.id
        FROM reviews r
        WHERE AI.IF(PROMPT('Does this review discuss the ending? {0}', r.body))
    """, dialect="bq")
    run = query.submit()
    print(run.id)

    for status in run.watch():
        print(status.state)

    result = run.result()
    print(result.collect().to_pydict())

submit() returns after the server saves the query. Save run.id to retrieve it later. run.result() waits for completion and downloads the result.

Use query.collect() instead if you only need the rows and want to wait for them.

Quail uploads tables registered with from_table, from_parquet, from_ipc, or from_dataset. Identical files are uploaded only once. For from_hf, the server downloads the dataset at the commit selected when you register it.

Server sessions accept SQL queries and these table providers. Custom providers and edited plans require a local session.

Inspect a query

Read its status

Use run.status() for the current status or run.watch() for updates, as shown above. A query passes through queued, planning, and running. It ends as succeeded, failed, interrupted, or cancelled.

The status includes phase with a description of the current activity and progress with execution counts. Either can be None before an update.

Read its plan and metrics

Read the plan after the server finishes planning:

status = run.status()
if status.plan is not None:
    print(status.plan["text"])

After execution, read the measured plan and report:

result = run.result()
print(result.explain())
print(result.report)

query.explain() is unavailable on a server session. See Query results and plans to read the plan and Metrics for report fields.

Inspect it in a browser

Open http://<host>:8642/ to list recent queries. Open http://<host>:8642/queries/<query_id> to inspect one query's status, progress, plan, errors, and the first 20 result rows. The page updates while the query runs.

If you set QUAIL_SERVER_TOKEN when starting the server, enter the same token when the browser page asks for it.

Reconnect to a query

Create another session with the same endpoint and retrieve the query using its saved ID. You do not need to register the tables again:

with quail.Session(config=config, endpoint=endpoint) as session:
    run = session.get_run("<saved-query-id>")
    result = run.result()
    print(result.collect().to_pydict())

You can set query_id in submit() if you need to retry a submission after losing its response. The same ID returns the existing query only if the SQL, inputs, configuration, and timeout match. Otherwise, the server raises QueryIdConflictError.

Stop a query or handle a failure

Call run.cancel() to cancel a query.

Set timeout_s in submit() to limit execution time. The timer starts when the query leaves the queue. The default is 1,000 seconds. Use quail-server --help to see the timeout settings.

If the query ends without success, run.result() raises QueryFailedError. Read run.status().error for the reason.

After a server restart, queued queries resume. Queries that were planning or running become interrupted and must be submitted again with a new ID. Keep the server data directory to retain completed results.

Use a repository checkout and an authenticated Modal CLI to deploy Quail Server on one H100 in the quail-engine app.

Create the access token secret before deployment:

modal secret create quail-server-token QUAIL_SERVER_TOKEN='replace-with-your-token'
modal deploy -m quail.server.modal_app 2>&1 | tee server-deploy.log

Set the same QUAIL_SERVER_TOKEN in your Python program's environment. Pass the endpoint URL printed by modal deploy to Session. For DiffusionGemma, include HF_TOKEN in the secret so the server can download its weights.

Inputs and results are saved on the quail-results volume under /results/quail-server/. After a restart, query status is restored from the latest saved database copy. Recent updates can be lost.

The container stops after 15 idle minutes. The next request starts another container, and its first query loads the model again.