Extending Quail
Observers
Observers let you run code after each physical node executes. The runner calls every registered observer once per node, in execution order.
When you need one
Most of the time, result.report and result.node_metrics have
everything you need after a query finishes (see
Metrics). Use an observer when you need
something to happen during execution, for example logging progress
or streaming metrics to an external system.
Protocol
class ExecutionObserver(Protocol):
name: str
def after_node(self, node, result) -> None: ...
def report(self) -> Mapping[str, Any]: ...after_nodereceives the physical node and its output after execution. It cannot modify values.reportreturns a dict that is attached toresult.report["observers"][name]after the query finishes.
Source:
quail/execution/runner.py.
Example
import logging
class ProgressObserver:
name = "progress"
def __init__(self):
self.log = []
def after_node(self, node, result):
logging.info("%s: %.2fs", node.node_id, result.metrics.wall_s)
self.log.append(node.node_id)
def report(self):
return {"nodes": self.log}Registration
Register the class on the session's registry. The runner creates a fresh instance for each query.
registry = quail.ExtensionRegistry.with_built_ins()
registry.register_observer(ProgressObserver)
session = quail.Session(config=config, registry=registry)
result = session.sql(..., dialect="snowflake").run()
result.report["observers"]["progress"] # {"nodes": ["ai_filter:r", ...]}