Observers

An execution observer is an extension that receives each operator's result. You can implement an observer when Quail's standard query report does not contain the information you need. The observer can read operator outputs and metrics, then add its own report to the query result.

Here we show how to implement an observer, register it with a session, and read its report.

Quail calls observers after model execution, while it prepares the query result. These calls do not provide progress updates during GPU execution.

Implement the observer

The example below records operator IDs and includes them in a report. You can define a class with name, after_node(), and report():

class CompletedOperators:
    name = "completed_operators"

    def __init__(self):
        self.node_ids = []

    def after_node(self, node, result):
        self.node_ids.append(node.node_id)

    def report(self):
        return {"node_ids": self.node_ids}

• name identifies the observer's report in the query result.
• after_node(node, result) receives a physical node and its NodeResult. You can read the outputs and metrics from result. Do not change the operator's outputs.
• report() returns a dictionary with the values you want to include in the query report.

See the ExecutionObserver interface.

Register the observer

Register the class with the extension registry. Quail creates a new instance for each query, so values collected for one query do not appear in the next query's report.

import quail

registry = quail.ExtensionRegistry.with_built_ins()
registry.register_observer(CompletedOperators)
config = quail.EngineConfig(model="qwen3-4b-fp8", device="h100-sxm")
session = quail.Session(config=config, registry=registry)

Read the report

Create a query with this session. After running it, call result.observer() with the observer class to read its report:

result = query.run()
print(result.observer(CompletedOperators))
session.close()

The report contains the node_ids list returned by report(). For Quail's standard execution measurements, see Metrics.