Reading List Multi-Table QA & Text-to-SQL over Data Lakes

A canonical text-to-SQL benchmark: 10,181 questions and 5,693 SQL queries over 200 databases spanning 138 domains. Designed to test cross-domain generalization — train and test databases are disjoint. The de facto evaluation set for nearly all text-to-SQL work, with execution accuracy rising from ~54% (2020) to over 90% (2023) as a clear progress narrative.

Pushes the text-to-SQL bar significantly higher than Spider: 12,751 question/SQL pairs over 95 databases (33 GB total), with dirty values, large schemas, and external knowledge required to answer many questions. Introduces "valid efficiency score" alongside execution accuracy. By 2025, leaderboard execution accuracy reached ~76% — substantially below Spider, exposing gaps between toy and realistic settings.

A comprehensive survey covering LLM prompt engineering, fine-tuning, retrieval-augmented, and agentic approaches. Catalogues benchmarks (Spider, BIRD, Spider 2.0, BIRD-Critic) and tracks the progression from single-pass SQL generation to multi-step pipelines. Distinguishes itself from earlier surveys with a systematic taxonomy and "Key Takeaways" boxes at the end of each section.

A multi-agent framework with four specialized agents: the Information Retriever extracts relevant data, the Schema Selector prunes large schemas, the Candidate Generator produces and iteratively refines SQL candidates, and the Unit Tester validates queries through LLM-based natural-language unit tests. Uses locality-sensitive hashing to retrieve database values from millions of rows. One of the most-cited baselines for schema-linking-heavy pipelines and a frequent comparison point for newer methods.

Combines bidirectional schema linking (forward and backward pruning), contextual augmentation, a binary mode-selection strategy, and multi-turn self-correction. Achieves 94% strict recall on schema linking while reducing input columns by 83% — concretely showing how much of the SQL-generation problem is really a retrieval problem.

Targets the truly massive-schema setting (thousands of tables). Reduces total schema to less than 3% on BIRD and less than 1% on FIBEN through retrieval-augmented filtering. Highlights that schema linking is a different problem at enterprise scale than on academic benchmarks.

Introduces a "restricted missing indicator" metric arguing that recall and precision alone don't capture schema-linking quality. Treats schema linking as a knapsack problem balancing inclusion against a tolerance budget for redundancy. A 1.6B model with KaSLA matches DeepSeek-V3 + SOTA schema-linking on Spider and BIRD.

Three specialized agents — a Selector for schema simplification, a Decomposer for sub-question generation, and a Refiner for self-correction — collaborating on text-to-SQL. Demonstrates that the right division of labor between LLM-driven components can substantially improve over single-pass generation.

Reformulates schema linking as an iterative, agent-driven exploration: the agent never sees the full schema, but dynamically expands the linked subset through actions. Achieves 97.4% strict recall on BIRD-Dev and 91.2% on Spider 2.0-Lite at sub-half the token cost of leading approaches. The strongest current case for agentic schema discovery.

Among the first to combine text-to-SQL with open-domain QA: a generative model takes both textual and tabular evidence and chooses to emit either a direct answer or a SQL query, executing the latter on the relevant database. Demonstrates that interpretable SQL generation helps especially for questions requiring complex reasoning.

An early RAG-style architecture specifically for open-domain table QA: retrieve candidate tables, then condition a generator on them. Sets the template that nearly every subsequent LLM-based system follows in some form.

A systematic analysis of LLMs paired with table retrievers on private tabular data lakes. Finds that answer generation fails primarily because retrieval doesn't provide the required tabular context — making table retrieval, not SQL synthesis, the binding constraint for open table QA at scale.

Argues that table retrieval is far from solved: existing methods retrieve individually relevant tables but fail when the answer requires joining them, and they assume key/foreign-key constraints are pre-given — which is exactly what's missing in data-lake settings. Proposes join-aware retrieval that ranks sets of tables by their joint relevance and joinability.

Frames multi-table retrieval as iterative search. Argues that exact methods like Mixed-Integer Programming guarantee coherence but don't scale, while greedy heuristics that optimize coverage often miss joinable sets. The iterative-search formulation offers better scalability, interpretability, and flexibility than either extreme.

Targets numerical multi-table QA over tens of thousands of tables with incomplete metadata. Decomposes the question, retrieves tables for each sub-question, and stitches together cross-table reasoning. Explicitly positioned against both text-to-SQL (which assumes a complete schema) and single-table QA (which can't combine evidence).

Argues that pure text-to-SQL is insufficient for realistic data-analytics tasks: many questions require reasoning, semantic operations, and tool use beyond SQL. Proposes TAG (Table-Augmented Generation) as a unifying framework — a useful conceptual reset before diving into the engineering of newer data-lake systems.

The first benchmark explicitly designed for text-to-SQL over data lakes. Evaluates three capabilities: resolving ambiguity and redundancy in NL questions, retrieving relevant tables from large heterogeneous lakes, and generating SQL pipelines that may span multiple databases. Built on 1,500 NL questions from a Beijing smart-city data service with 31 government departments' data integrated. Questions and queries are significantly longer and more complex than Spider or BIRD.

A practitioner-oriented analysis of why Spider/BIRD progress doesn't translate to enterprise settings: large schemas, semantic ambiguity, dirty data, and operational constraints. Useful framing for understanding the gap between benchmark numbers and real-world deployment.

600 real-world workflow problems sourced from BigQuery, Snowflake, and PostgreSQL deployments. Specifically targets dialect-specific SQL, schema scale, and external-knowledge integration. Top systems score ~35% — a dramatic drop from Spider's 90%+, demonstrating how much of recent progress was overfitted to the benchmark.

Moves beyond SQL generation to SQL diagnosis and repair, built on authentic user-reported database issues. Tests whether systems can identify what's wrong with a SQL query, not just write one — a capability central to deploying these systems in real settings.

A skeptical look at NL2SQL evaluation. Examines how systems respond to paraphrased NL queries and finds substantial accuracy degradation on rewordings of the same question — a robustness gap most benchmarks don't measure.