News · Paper · Why TACO? · Dataset · Examples · Quick Start · Docs
⭐️ If you find TACO helpful, a star on GitHub would mean a lot to us — thanks for your support!
| Date | Update |
|---|---|
| Jun 2026 | 📄 Paper released on arXiv:2606.14201 |
| 2026 | 🎉 TACO has been accepted to VLDB 2026! |
TACO (Text-to-SQL with Ambiguous and Cross-database Open-domain queries) is a benchmark for real-world data-lake Text-to-SQL. Unlike Spider or BIRD — where the target database is known and schemas are clean — TACO asks models to cope with vague user questions, unspecified databases, and queries spanning multiple heterogeneous databases.
📄 Paper: arXiv:2606.14201 · 💻 Code: GitHub
Existing benchmarks largely assume a single, well-defined schema. In practice, users query open data lakes with messy intent and weak cross-source relationships. TACO fills this gap with three stress tests:
| Challenge | What makes it hard |
|---|---|
| 💬 Ambiguous NL | Redundant context, implicit constraints, and vague terms that do not map 1:1 to SQL |
| 🔍 Open-domain retrieval | The target database is not given — systems must find relevant tables across domains |
| 🔗 Cross-database SQL | A single question may require JOIN or UNION across 2–4 databases with weak keys |
What you get
- 📦 ~14,500 high-quality Text-to-SQL instances with executable gold SQL and validated results
- 🗄️ 46 databases across finance, healthcare, transportation, housing, government, and more
- 🌏 Two regional subsets — TACO-Beijing (24 DBs) and TACO-US (22 DBs)
- 📏 Standard evaluation splits and a baseline / ablation suite (execution accuracy as the primary metric)
The dataset is not included in git. Download from either source:
| Source | Link |
|---|---|
| Hugging Face (recommended) | Akanezora/TACO-Benchmark |
| Google Drive | TACO-Benchmark.zip |
# Hugging Face
from huggingface_hub import snapshot_download
snapshot_download("Akanezora/TACO-Benchmark", repo_type="dataset", local_dir="./taco-benchmark")# Google Drive (via project CLI)
taco data download && taco data verify| Subset | Databases | Single-DB SQL | Single-DB NL | Cross-DB SQL |
|---|---|---|---|---|
| TACO-Beijing | 24 | 4,028 | 5,587 | 466 |
| TACO-US | 22 | 3,990 | 3,990 | 429 |
| Total | 46 | 8,018 | 9,577 | 895 |
| Type | Share | ~Count |
|---|---|---|
| Single-database | 80.5% | ~11,700 |
| 2-database cross-DB | 15.0% | ~2,175 |
| 3-database cross-DB | 4.4% | ~638 |
| 4-database cross-DB | 0.1% | ~15 |
Format details and directory layout: docs/DATASET.md
The user asks for summaries, comparisons, and deviation analysis — but the gold SQL is a focused filter on one year:
"I need to verify the government fund budget revenue for the entire year of 2018. Please help me summarize the budget revenue and actual received amounts for all government fund budget projects in 2018… find projects with large deviations… This data is very important for annual settlement review…"
SELECT "finance_bureau_budget_execution_report"."ProjectName",
"finance_bureau_budget_execution_report"."BudgetRevenue2018",
"finance_bureau_budget_execution_report"."Year"
FROM "finance_bureau_budget_execution_report"
WHERE "finance_bureau_budget_execution_report"."Year" = '2018'Takeaway: models must extract the core intent from noisy, redundant NL — not over-generate SQL for every phrase the user mentions.
A single question spans two databases with an implicit join key:
"Query the construction project completion filing information from the Housing database, join with the public toilet information from the Life Services database, count the number of public toilets for each project name, and sort by the number of public toilets in descending order."
SELECT "Housing"."construction_project_completion_filing"."ProjectName",
COUNT("LifeServices"."public_toilet_info"."SequenceNumber") AS "ToiletCount"
FROM "Housing"."construction_project_completion_filing"
JOIN "LifeServices"."public_toilet_info"
ON "Housing"."construction_project_completion_filing"."SequenceNumber"
= "LifeServices"."public_toilet_info"."SequenceNumber"
GROUP BY "Housing"."construction_project_completion_filing"."ProjectName"
ORDER BY "ToiletCount" DESCTakeaway: systems must plan across databases, align schemas, and produce valid multi-DB SQL (e.g., ATTACH DATABASE in SQLite).
More examples (open-domain retrieval, UNION, 3–4 DB queries): docs/EXAMPLES.md
git clone https://github.com/Akanezora0/TACO-Benchmark.git && cd TACO-Benchmark
python scripts/setup_env.py && source .venv/bin/activate
# Download data from Hugging Face (see Dataset section) or: taco data download && taco data verify
taco eval run --model gpt-4o --dataset beijing # baseline evaluationSetup, API configuration, and troubleshooting: docs/INSTALL.md · Minimal repro script: examples/quick_eval.sh
We provide baseline experiments (GPT-4o, DIN-SQL, CodeS, …), TACO-SQL ablations, and execution-accuracy (EX) evaluation. See docs/EXPERIMENTS.md and experiments/README.md.
| Doc | Contents |
|---|---|
| docs/DATASET.md | Download, formats, directory layout |
| docs/EXAMPLES.md | Full challenge examples with NL/SQL pairs |
| docs/EXPERIMENTS.md | Baselines, ablations, CLI reference |
| docs/INSTALL.md | Environment setup |
| docs/GENERATION.md | Data regeneration pipeline (optional) |
| docs/ARCHITECTURE.md | Repository layout |
@article{deng2026taco,
title = {TACO: A Benchmark for Open-Domain Text-to-SQL with Ambiguous and Cross-Database Queries},
author = {Deng, Chao and Fan, Ju and Luo, Yuyu and Xue, Qinliang and Fan, Meihao and Zhang, Yuxin and Zhang, Min and Jia, Xiaofeng and Zhang, Jing and Du, Xiaoyong},
journal = {arXiv preprint arXiv:2606.14201},
year = {2026},
url = {https://arxiv.org/abs/2606.14201}
}⭐️ Enjoying TACO? Give us a star if you'd like to support the project!