Reading List Table Union Search in Data Lakes

Defines table union search as finding the top-k tables whose columns are unionable with a query table's columns. Introduces three attribute-level unionability statistics — set, semantic (entity-based), and natural-language (embedding-based) — combined into an ensemble score. Releases the original TUS benchmark, the de facto evaluation set for nearly all follow-on work.

Combines five column-pairwise similarity measures (attribute names, formats, word embeddings, distributions, domain) into a multi-criteria distance function, supporting both unionable and joinable discovery on the same indexing substrate. A common comparison baseline.

Pre-trains a column encoder with multi-column contrastive learning so that columns within the same table inform each other's embedding. Cosine similarity becomes the column unionability score; combined with a filter-and-verify framework and HNSW indexing for sub-linear search. Reports +6.8 MAP over prior baselines and becomes a standard 2024–25 reference point.

Frames union search as an unsupervised representation-learning problem with a contrastive objective that is aware of the downstream search/index structure. Tightly couples training and retrieval so that learned embeddings are good not just intrinsically but for the specific top-k nearest-neighbor query.

Two tables are unionable when their columns share both domain semantics and binary relationships. SANTOS retrieves relationships from an existing KB (YAGO) and synthesizes one from the data lake itself when KB coverage is thin. Releases relabeled and new benchmarks; outperforms the original TUS pipeline by a wide margin.

Argues column-centric methods miss holistic table semantics and ranking quality suffers. Generates a single table embedding via attentive table encoding with positive-pair construction and two-pronged negative sampling (avoiding latent positives, mining hard negatives). Retrieval uses an adaptive candidate-pool method that exploits the table-score distribution; column matching only happens in a final verification step.

An LLM-based framework that operates directly on table content without per-lake fine-tuning, enabling search across multiple data lakes. Comes with TUSBench, a benchmarking environment integrated in the same framework, oriented toward reproducibility and joint evaluation of accuracy and runtime.

Reframes the unit of discovery from tables to tuples: given a query table, return a set of unionable tuples from across the lake that is both relevant and maximally diverse — so the union actually adds new information rather than duplicating what the analyst already has. Formulates the problem as joint optimization over unionability and novelty/diversity, with retrieval and ranking algorithms designed for that combined objective.

Asks a complementary question at the table level: which tables in the lake offer content that is genuinely novel with respect to a query table? Existing unionability search rewards tables that look similar to the query — exactly the wrong objective if the goal is to enrich. Defines novelty for table search and proposes retrieval methods that surface tables most likely to extend rather than echo the query.

Uses LLMs to generate evaluation benchmarks for table union search — addressing the chronic scarcity of large, well-labeled benchmarks. Complements rather than replaces SANTOS and TUS benchmarks; useful both as a benchmark-generation method and as a study of LLMs' notion of unionability.

Two new tabular benchmarks (small and large real data lakes) plus a relabeled version of the original TUS benchmark that incorporates inter-column relationships. The standard set for evaluating relationship-aware methods.

A unified, large-scale benchmark covering both joinable and unionable table search. 16M+ tables, 10K+ labeled queries, with reference implementations for major baselines — the natural starting point for project evaluations and the most reliable way to compare modern methods head-to-head.

Goes beyond search: once unionable tables are retrieved, how do you actually combine them? Studies alignment of columns and resolution of heterogeneity across union candidates — a useful framing for "what comes after the search".

Introduces a new problem that sits downstream of table discovery: given a source table, find and integrate a set of data lake tables that reproduce it as closely as possible. Defines an error-aware instance similarity measure and supports a richer query language than SPJ — including unions, outerjoins, subsumption, and complementation. A useful complement to search-focused papers when thinking about what data lakes need to do after a candidate set is retrieved.