Disaggregating RocksDB: A Production Experience
Proceedings of the ACM on Management of Data · 13.06.2023 · 1 (2) · 1-24
DOI: 10.1145/3589772 ↗Kurz gefasst
Fachartikel in der Zeitschrift Proceedings of the ACM on Management of Data, online erschienen am 20.06.2023. Verfasst von Siying Dong und 18 weiteren Autor:innen. Die Arbeit wurde bisher 25-mal zitiert (Crossref) und nennt 42 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).
Abstract (Originalsprache)
As in the general industry, there is a trend in Meta's data centers to migrate data from locally attached SSDs to cloud storage. We extended RocksDB [26], a widely used open-source storage engine designed and built for local SSDs, to leverage disaggregated storage. RocksDB's design, such as its data and log files' access patterns, makes an append-only distributed file system a desirable underlying storage. At Meta, we built disaggregated RocksDB using Tectonic File System [35], which so far had mainly been used for our data warehouse and blob storage stacks. We identified that metadata overhead and tail latencies were Tectonic's major performance gaps and addressed them accordingly. We improved the reliability, performance and other requirements with both general and customized optimizations to the core engine in RocksDB. We also took the time to deeply understand the common challenges presented by applications running on RocksDB and implemented enhancements to address them. This architecture enabled RocksDB to adapt to a more distributed architecture for performance enhancements.
Zitieren
APA
Dong, S., P, S. S., Pan, S., Ananthabhotla, A., Ekambaram, D., Sharma, A., Dayal, S., Parikh, N. V., Jin, Y., Kim, A., Patil, S., Zhuang, J., Dunster, S., Mahajan, A., Chelluri, A., Datye, C., Santana, L. V., Garg, N., & Gawde, O. (2023). Disaggregating RocksDB: A Production Experience. Proceedings of the ACM on Management of Data, 1(2), 1-24. https://doi.org/10.1145/3589772
Vancouver
Dong S, P SS, Pan S, Ananthabhotla A, Ekambaram D, Sharma A, et al. Disaggregating RocksDB: A Production Experience. Proceedings of the ACM on Management of Data. 2023;1(2):1-24. doi:10.1145/3589772
BibTeX
@article{Dong2023,
author = {Dong, Siying and P, Shiva Shankar and Pan, Satadru and Ananthabhotla, Anand and Ekambaram, Dhanabal and Sharma, Abhinav and Dayal, Shobhit and Parikh, Nishant Vinaybhai and Jin, Yanqin and Kim, Albert and Patil, Sushil and Zhuang, Jay and Dunster, Sam and Mahajan, Akanksha and Chelluri, Anirudh and Datye, Chaitanya and Santana, Lucas Vasconcelos and Garg, Nitin and Gawde, Omkar},
title = {Disaggregating RocksDB: A Production Experience},
journal = {Proceedings of the ACM on Management of Data},
year = {2023},
volume = {1},
number = {2},
pages = {1--24},
doi = {10.1145/3589772}
}
EUVIMED