Overview

Quail is structured like a traditional query engine: it parses a query into a logical plan, optimizes it, lowers it to a physical plan, and executes the physical plan. Execution is pull-based (Volcano-style), with documents pipelining through operators in large batches sized to saturate the GPU. The two performance goals are to keep the GPU busy (minimize idle time between forward passes) and to minimize KV regret (document state that was evicted from GPU memory and needs to be recomputed).

Query lifecycle

1. Quail parses the user's SQL (or Python builder query) into a logical plan.
2. The session reads Arrow batches from each registered table and tokenizes the document columns.
3. The planner proposes one or more physical plans with cost estimates. The cheapest one is selected.
4. The runner executes the physical graph. Model nodes run on the GPU. Relational nodes run with Arrow on the CPU.
5. The query returns the matching rows as a pyarrow.Table and an execution report with timing and token counts.

Packages

Each package imports only from the ones below it.