Revisiting Task-Oriented Dataset Search in the Era of Large Language Models: Challenges, Benchmark, and Solution
Proceedings of the VLDB Endowment · 01.01.2026 · 19 (5) · 973-986
DOI: 10.14778/3796195.3796209 ↗Kurz gefasst
Fachartikel in der Zeitschrift Proceedings of the VLDB Endowment, online erschienen am 05.05.2026. Verfasst von Zixin Wei und 5 weiteren Autor:innen. Die Arbeit nennt 40 Literaturangaben.
Abstract (Originalsprache)
The search for suitable datasets is the critical "first step" in data-driven research, but it remains a great challenge. Researchers often need to search for datasets based on high-level task descriptions. However, existing search systems struggle with this task due to ambiguous user intent, task-to-dataset mapping and benchmark gaps, and entity ambiguity. To address these challenges, we introduce KATS, a novel end-to-end system for task-oriented dataset search from unstructured scientific literature. KATS consists of two key components, i.e., offline knowledge base construction and online query processing. The sophisticated offline pipeline automatically constructs a high-quality, dynamically updatable task-dataset knowledge graph by employing a collaborative multi-agent framework for information extraction, thereby filling the task-to-dataset mapping gap. To further address the challenge of entity ambiguity, a unique semantic-based mechanism is used for task entity linking and dataset entity resolution. For online retrieval, KATS utilizes a specialized hybrid query engine that combines vector search with graph-based ranking to generate highly relevant results. Additionally, we introduce CS-TDS, a tailored benchmark suite for evaluating task-oriented dataset search systems, addressing the critical gap in standardized evaluation. Experiments on our benchmark suite show that KATS significantly outperforms state-of-the-art retrieval-augmented generation frameworks in both effectiveness and efficiency, providing a robust blueprint for the next generation of dataset discovery systems.
Zitieren
APA
Wei, Z., Guo, Y., Li, J., Han, X., Jin, X., & Ma, C. (2026). Revisiting Task-Oriented Dataset Search in the Era of Large Language Models: Challenges, Benchmark, and Solution. Proceedings of the VLDB Endowment, 19(5), 973-986. https://doi.org/10.14778/3796195.3796209
Vancouver
Wei Z, Guo Y, Li J, Han X, Jin X, Ma C. Revisiting Task-Oriented Dataset Search in the Era of Large Language Models: Challenges, Benchmark, and Solution. Proceedings of the VLDB Endowment. 2026;19(5):973-986. doi:10.14778/3796195.3796209
BibTeX
@article{Wei2026,
author = {Wei, Zixin and Guo, Yucan and Li, Jinyang and Han, Xiaolin and Jin, Xiaolong and Ma, Chenhao},
title = {Revisiting Task-Oriented Dataset Search in the Era of Large Language Models: Challenges, Benchmark, and Solution},
journal = {Proceedings of the VLDB Endowment},
year = {2026},
volume = {19},
number = {5},
pages = {973--986},
doi = {10.14778/3796195.3796209}
}
EUVIMED