Logical plans

A logical plan describes the operations a query requests and the data they read. It does not specify GPU batches or attention settings. The SQL frontend binds a query, and a placement rule builds the plan from what it bound (see From SQL to a plan). The optimizer then rewrites it before building a physical plan.

This page describes the plan representation for contributors working on query syntax, validation, or optimizer rules.

Plan structure

The plan is a tree. Each node represents an operation, and its children are the operations that supply its inputs. Data flows from scans toward the root.

For example, imagine a query that classifies reviews by sentiment and returns only negative reviews. Its logical plan has these operations:

Result
  Project: r.id, sentiment
    Filter: sentiment IN ('negative')
      SemanticClassify: sentiment
        Scan: reviews as r

SemanticClassify calls the model and adds a label column. Filter tests that column without calling the model. Keeping the two operations separate lets the planner reason about their placement and cost.

From SQL to a plan

The SQL frontend resolves each name to a table, a column, or a model call. It does not decide where anything runs. It writes the FROM, JOIN, and WHERE part of the query as one tree:

Filter: AI.IF('Review {0} discusses product {1}') AND AI.IF('This review is negative') AND r.stars > 3
  Join on r.asin = p.asin
    Scan: reviews as r
    Scan: products as p

The Filter holds every ON and WHERE part as written, the ON parts first. quail.frontend.sql.bind_relation returns this tree for one SELECT.

place_relation in quail/logical/naive.py then places each part:

A test of source columns of one table goes into that table's Scan.
An AI.IF or AI.SCORE test of one table goes into its SemanticFilter.
An AI.IF over two tables becomes a SemanticJoin, with the ON equalities of the tables it brings in.
An EXISTS or NOT EXISTS over a model call becomes a gate join.
A test of a label or an answer goes right after the operator that computes the column; any other part runs on the result rows.

LogicalPlanBuilder then puts each table's operators in order, and its project() adds the result nodes.

The node protocol

All logical nodes implement this protocol:

class LogicalNode(Protocol):
    type_name: ClassVar[str]

    def children(self) -> tuple[LogicalNode, ...]: ...
    def expressions(self) -> tuple[Any, ...]: ...
    def output_schema(self) -> tuple[ColumnRef, ...]: ...
    def validate(self) -> None: ...
    def with_children(self, children) -> LogicalNode: ...
    def with_expressions(self, expressions) -> LogicalNode: ...
    def explain_fields(self) -> dict[str, Any]: ...

children() returns the node's inputs.
expressions() returns the expressions attached to the operation. A SemanticFilter holds FilterPredicate objects, which include a predicate and a selectivity hint. A Project holds output expressions.
output_schema() describes the columns available after the operation.
validate() checks the node's constraints.
explain_fields() supplies fields for the plan's text representation.

Nodes are frozen dataclasses. Rewrites use with_children(), with_expressions(), or dataclasses.replace() to return new nodes. LogicalPlan.walk() visits children before their parents.

Node types

Node	Meaning
`Scan(provider, alias, column)`	Read document text from a registered table.
`SemanticFilter(input, predicates)`	Evaluate model predicates over one table and keep passing rows.
`Join(left, right, on)`	Form pairs from two inputs. Optional column equalities restrict the pairs. Without `on`, it forms every pair.
`SemanticJoin(input, predicate, semantics, selectivity, anchor)`	Evaluate a model predicate over the pairs from a `Join`. The semantics are `full`, `exists`, or `anti`.
`SemanticClassify(input, call, name)`	Evaluate an `AI.CLASSIFY` call and add the label column `name`.
`SemanticExtract(input, call, name)`	Extract one answer field and its character positions from each document.
`Filter(input, condition, selectivity)`	Keep rows that satisfy a condition on a label, extracted answer, or answer position.
`Apply(input, function, kind, ids, columns, aliases)`	Run a registered Python function on input rows or pairs.
`Project(input, columns, structs)`	Select values and assemble extraction structs.
`Aggregate(input, keys, aggregates, output, having)`	Group projected rows and compute aggregate values.
`Window(input, calls, output)`	Add window values to projected rows, preserving individual rows.
`Result(input, limit, order, offset, distinct)`	Set result order, remove duplicates, and apply an offset and limit. This is the root node.

A query joining three or more tables uses one SemanticJoin for each binary join. A classification may read one document or both documents of a joined pair. Conditions on labels of joined pairs are unsupported.

A window query places Window between Project and Result. The projection supplies source columns and AI outputs used by the window. The window adds values such as a previous row's label or a running sum. Result specifies the final row order and limit.

Expressions

Expressions describe the values an operator reads, computes, or tests. A value expression reads a column, accesses a struct field, or applies a function without calling the model. A ModelCall describes an AI operation.

Expression	Value
`ColumnRef(alias, provider, column)`	A reference to a table column.
`GetField(input, field)`	A field read from a struct.
`ScalarFunction(function, args)`	A value computed for each row, such as text length.
`RegularPredicate(expression, comparison, value)`	A condition on a value, such as `length(r.body) < 40`.
`ModelCall(prompt, kind, labels, descriptions)`	An AI operation. The kind is `boolean`, `score`, `label`, or `extract`.
`Alias(expression, name)`	An expression returned under a specified column name.
`WindowCall(function, argument, partition, order, name)`	A window function with its input column, partition keys, order keys, and output name.

For example, LENGTH(r.details.body) reads the details column, selects its body field, and computes the text length. Its expression is:

ScalarFunction("length", (
    GetField(ColumnRef("r", "reviews", "details"), "body"),
))

Operators use these expressions as follows:

Scan.column specifies the text to tokenize. It can read a source column directly or use a value expression, such as a struct field.
SemanticFilter and SemanticJoin test a boolean ModelCall or a RegularPredicate that compares an AI.SCORE call with a threshold.
Filter tests a label, extracted answer, or answer position with a RegularPredicate. The condition does not call the model again.
Project.columns specifies the returned values and their names.

LogicalPlan.operators() collects the tree's operations for planning. It includes scans, predicates and joins in written order, Python functions, score, classification, and extraction calls, and conditions on their answers.

Prompt binding

Prompt binding converts a user's template and column references into the layout the model receives. Quail places document text before the question so questions about the same document can reuse its KV. KV is the key and value vectors saved for previously processed tokens.

A template is therefore not treated as a plain string substitution. For example, PROMPT('Does {0} praise the acting?', r.body) identifies r.body as the document, but Quail sends that document before the question.

Filters

A bound filter prompt contains, in order:

A document heading.
The document text.
The task instruction and question.
The answer cue.

For example, for a review with body "The acting was excellent.", this call:

PROMPT('Does this review praise the acting? {0}', r.body)

uses this layout:

DOCUMENT:
The acting was excellent.
You are performing a data processing task.
Evaluate TRUE or FALSE for the following question:
Does this review praise the acting?
ANSWER:

Joins

The join layout places the anchor document first. The anchor is the side whose KV remains available while Quail evaluates its partners.

The prompt contains, in order:

A heading and the anchor document.
The anchor frame, which identifies the anchor and states the question.
A heading and the partner document.
The answer cue.

The frame is fixed for all partners of an anchor. In the Quail executor, the first partner heading is also stored with that frame, once per anchor. Each pair then adds its partner document and answer cue. See Join execution.

Classification

A classification prompt includes the document, question, and allowed categories. Quail prepares both category text and letter choices when one distinct token is available for each letter. The label scoring rule chooses which version to use.

For example, suppose reviews.body contains "The acting was excellent." This call supplies two sentiment categories:

AI.CLASSIFY(
    PROMPT('What sentiment does this review express? {0}', r.body),
    ARRAY['positive', 'negative'])

The version with letter choices has this layout before model chat formatting is added:

DOCUMENT:
The acting was excellent.

Answer with the letter of exactly one of the categories below for the following question: What sentiment does this review express?

Categories:
- A: positive
- B: negative
ANSWER:

The document precedes the question and category list, so another question can reuse the same document KV. When the model selects A, Quail returns the category text positive in the result column.

Tokenization

Quail tokenizes documents separately from fixed prompt text. The executor joins the token sequences without tokenizing the complete text again. This lets it reuse document tokens across queries. A bound prompt is text. The planner lowers it once per query with the session's tokenizer (quail.planner.prompt_lowering.lower_prompt), and the lowered prompt holds the fixed token IDs and counts the cost model and the executor read.

For a score over one document, the input is:

tokenize(before_document) + tokenize(document) + tokenize(after_document)

Separate tokenization can produce different tokens from tokenizing the complete string. A token can span the boundary between fixed text and the document, such as a space and the first word. Splitting at that boundary prevents the combination and can change model scores.

When comparing implementations, use identical token IDs to compare model execution. A comparison against an implementation that tokenizes the complete string also measures the effect of tokenization.

Logical rules

Logical rules rewrite the plan before physical planning. See the optimizer for the built-in rules and Rules for the extension interface.

Logical plans

On this page