EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

Data formats in analytical DBMSs: performance trade-offs and future directions

Chunwei Liu, Anna Pavlenko, Matteo Interlandi, Brandon Haynes

The VLDB Journal · 2025-05 · 34 (3)

DOI: 10.1007/s00778-025-00911-1 ↗

Kurz gefasst

Fachartikel in der Zeitschrift The VLDB Journal, online erschienen am 19.03.2025. Verfasst von Chunwei Liu und 3 weiteren Autor:innen. Die Arbeit wurde bisher 7-mal zitiert (Crossref) und nennt 79 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: Massachusetts Institute of Technology.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

This paper evaluates the suitability of Apache Arrow, Parquet, and ORC as formats for subsumption in an analytical DBMS. We systematically identify and explore the high-level features that are important to support efficient querying in modern OLAP DBMSs and evaluate the ability of each format to support these features. We find that each format has trade-offs that make it more or less suitable for use as a format in a DBMS and identify opportunities to more holistically co-design a unified in-memory and on-disk data representation. Notably, for certain popular machine learning tasks, none of these formats perform optimally, highlighting significant opportunities for advancing format design. Our hope is that this study can be used as a guide for system developers designing and using these formats, as well as provide the community with directions to pursue for improving these common open formats.

Förderung

  • Massachusetts Institute of Technology

Zitieren

APA

Liu, C., Pavlenko, A., Interlandi, M., & Haynes, B. (2025). Data formats in analytical DBMSs: performance trade-offs and future directions. The VLDB Journal, 34(3). https://doi.org/10.1007/s00778-025-00911-1

Vancouver

Liu C, Pavlenko A, Interlandi M, Haynes B. Data formats in analytical DBMSs: performance trade-offs and future directions. The VLDB Journal. 2025;34(3). doi:10.1007/s00778-025-00911-1

BibTeX
@article{Liu2025,
  author = {Liu, Chunwei and Pavlenko, Anna and Interlandi, Matteo and Haynes, Brandon},
  title = {Data formats in analytical DBMSs: performance trade-offs and future directions},
  journal = {The VLDB Journal},
  year = {2025},
  volume = {34},
  number = {3},
  doi = {10.1007/s00778-025-00911-1}
}
Kostenlos starten →