Can LLM agents reliably answer business questions over real company databases, not just text-to-SQL benchmarks?

Not yet, once the data is spread across systems and messy. Our DAB benchmark has 104 questions over 17 datasets in PostgreSQL, MongoDB, SQLite, and DuckDB, based on a study of six enterprises building data agents. The best agent, GPT-5.5 in Codex, answered 57% correctly on the first try, and 70% within five tries.

Sources: Can AI Agents Answer Your Data Questions? A Benchmark for Data Agents (Preprint)

More questions on this topic

All questions about using AI for data and knowledge work