Logical plans

A logical plan describes what a query does. Both front ends (SQL and the Python builder) produce one through LogicalPlanBuilder, so they always generate the same plan for the same query.

The node protocol

All logical nodes implement this protocol:

class LogicalNode(Protocol):
    type_name: ClassVar[str]

    def children(self) -> tuple[LogicalNode, ...]: ...
    def expressions(self) -> tuple[Any, ...]: ...
    def output_schema(self) -> tuple[ColumnRef, ...]: ...
    def validate(self) -> None: ...
    def with_children(self, children) -> LogicalNode: ...
    def with_expressions(self, expressions) -> LogicalNode: ...
    def explain_fields(self) -> dict[str, Any]: ...

children() returns the node's inputs in the plan tree. For example, a SemanticFilter's child is its Scan, and a Project's child is the filter or join below it. expressions() returns the expressions attached to the node: for a SemanticFilter these are its FilterPredicate objects, each holding a predicate expression and a selectivity hint, and for a Project these are the output columns.

Nodes are frozen dataclasses. Since they are immutable, with_children and with_expressions return a new node with the specified fields replaced. Logical rules (see below) use these methods to transform the plan. LogicalPlan.walk() visits children before parents.

Node types

Expressions

The AI functions of the language are expressions, not operators. An operator holds expressions and does not change when a new AI function arrives.

A predicate on a SemanticFilter or SemanticJoin answers yes or no: a boolean ModelCall, or a Compare around a score call. Project.columns holds ColumnRef and Alias values.

LogicalPlan.operators() is the one walk that turns the tree into what the planner reads: the scans, each alias's filter predicates in written order, the joins in written order, and the applies.

Prompt binding

When the parser sees a PROMPT(...), it creates a Prompt object that rearranges the template into a fixed layout.

All model task instructions include "You are performing a data processing task." AI.IF filters and joins place it before the question in their completion prompt. AI.SCORE places it in the reranker's system message. Quail and the stock vLLM backend use the same instruction text.

For a filter, bind_prompt produces:

1. Preamble: DOCUMENT:\n
2. Document text: the value of the column for this row.
3. Question: the instruction from the template, wrapped in an evaluation prompt.
4. Answer cue: \nANSWER:

PROMPT('Does this review mention a positive aspect?\n\n{0}', r.body)

Sent to the model as:

DOCUMENT:
A beautiful film with outstanding performances...
You are performing a data processing task.
Evaluate TRUE or FALSE for the following question:
Does this review mention a positive aspect?
ANSWER:

For a join, bind_join_prompt produces:

1. Preamble: DOCUMENT {0}:\n
2. Anchor document text (e.g., r.body).
3. Anchor frame: a label identifying the document, followed by the full question. This stays in the anchor's KV.
4. Partner label and document (DOCUMENT {1}:\n + e.g., a.aspect).
5. Answer cue: \nANSWER:

The anchor's preamble, document, and frame are computed once. For each partner, only the partner label, partner document, and answer cue are new.

PROMPT('Does the review discuss this aspect?\n\n{0}\n\n{1}', r.body, a.aspect)
anchor: r

Sent to the model as:

DOCUMENT {0}:                                      ← anchor (r.body)
A beautiful film with outstanding performances...
                                                    ← anchor frame (computed once)
(The document above is DOCUMENT {0}.)
You are performing a data processing task.
Evaluate TRUE or FALSE for the following question:
Does the review discuss this aspect?
Review: {0}
Aspect: {1}

DOCUMENT {1}:                                      ← partner (a.aspect)
the acting
ANSWER:

In both cases, the document is placed before the question. This way, the document's KV is the same regardless of which question follows, which is what makes KV rewind possible.

Token counts and ids are stored on the Prompt so the planner and executor do not re-tokenize.

Tokenization

Quail tokenizes documents separately from the fixed prompt text so document tokens can be reused across queries. The executor combines the token sequences without tokenizing the combined text again. Both AI.IF and AI.SCORE use this approach. AI.SCORE uses the reranker's prompt format, with fixed parts tokenized by _token_parts.

For a single-document score, the input is:

tokenize(before_document) + tokenize(document) + tokenize(after_document)

This can differ from tokenizing the complete prompt string. A token can span a boundary between parts, such as a space before the document and its first word. Separate tokenization prevents that combination and can change model scores.

When comparing model implementations, pass the same token ids to both. Comparing with a complete prompt string also tests this tokenization difference.

Logical rules

class LogicalOptimizerRule(Protocol):
    name: str
    def rewrite(self, root, context) -> LogicalNode | None: ...

A rule gets the whole plan and returns a new root, or None if nothing changed. apply_logical_rules runs the rules in order and repeats until a pass changes nothing, up to MAX_PASSES.

Quail includes one built-in logical rule:

• Projection pushdown (ProjectionPushdown): rewrites each Scan to load only the columns the query actually uses.

Other optimizations like filter ordering and join ordering currently happen in the planner, because they depend on cost estimates from the model and device specs. They could also be expressed as logical rules.

Filter placement (filters above their scans, before joins) is handled during plan construction in LogicalPlanBuilder.

Extensions can register additional rules through registry.register_logical_rule(...).