EUVIMEDEUVIMED entdecken
Crossref

Pneuma : Leveraging LLMs for Tabular Data Representation and Retrieval in an End-to-End System

Muhammad Imam Luthfi Balaka, David Alexander, Qiming Wang, Yue Gong, Adila Krisnadhi, Raul Castro Fernandez

Proceedings of the ACM on Management of Data · 17.06.2025 · 3 (3) · 1-28

DOI: 10.1145/3725337 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Proceedings of the ACM on Management of Data, online erschienen am 18.06.2025. Verfasst von Muhammad Imam Luthfi Balaka und 5 weiteren Autor:innen. Die Arbeit wurde bisher 19-mal zitiert (Crossref) und nennt 64 Literaturangaben.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Finding relevant tables among databases, lakes, and repositories is the first step in extracting value from data. Such a task remains difficult because assessing whether a table is relevant to a problem does not always depend only on its content but also on the context, which is usually tribal knowledge known to the individual or team. While tools like data catalogs and academic data discovery systems target this problem, they rely on keyword search or more complex interfaces, limiting non-technical users' ability to find relevant data. The advent of large language models (LLMs) offers a unique opportunity for users to ask questions directly in natural language, making dataset discovery more intuitive, accessible, and efficient. In this paper, we introduce Pneuma , a retrieval-augmented generation (RAG) system designed to efficiently and effectively discover tabular data. Pneuma leverages large language models (LLMs) for both table representation and table retrieval. For table representation, Pneuma preserves schema and row-level information to ensure comprehensive data understanding. For table retrieval, Pneuma augments LLMs with traditional information retrieval techniques, such as full-text and vector search, harnessing the strengths of both to improve retrieval performance. To evaluate Pneuma , we generate comprehensive benchmarks that simulate table discovery workload on six real-world datasets including enterprise data, scientific databases, warehousing data, and open data. Our results demonstrate that Pneuma outperforms widely used table search systems (such as full-text search and state-of-the-art RAG systems) in accuracy and resource efficiency.

Zitieren

APA

Balaka, M. I. L., Alexander, D., Wang, Q., Gong, Y., Krisnadhi, A., & Fernandez, R. C. (2025). Pneuma : Leveraging LLMs for Tabular Data Representation and Retrieval in an End-to-End System. Proceedings of the ACM on Management of Data, 3(3), 1-28. https://doi.org/10.1145/3725337

Vancouver

Balaka MIL, Alexander D, Wang Q, Gong Y, Krisnadhi A, Fernandez RC. Pneuma : Leveraging LLMs for Tabular Data Representation and Retrieval in an End-to-End System. Proceedings of the ACM on Management of Data. 2025;3(3):1-28. doi:10.1145/3725337

BibTeX
@article{Balaka2025,
  author = {Balaka, Muhammad Imam Luthfi and Alexander, David and Wang, Qiming and Gong, Yue and Krisnadhi, Adila and Fernandez, Raul Castro},
  title = {Pneuma : Leveraging LLMs for Tabular Data Representation and Retrieval in an End-to-End System},
  journal = {Proceedings of the ACM on Management of Data},
  year = {2025},
  volume = {3},
  number = {3},
  pages = {1--28},
  doi = {10.1145/3725337}
}
Kostenlos starten →