Adaptive Low-Precision Training for Embeddings in Click-Through Rate Prediction
Proceedings of the AAAI Conference on Artificial Intelligence · 26.06.2023 · 37 (4) · 4435-4443
DOI: 10.1609/aaai.v37i4.25564 ↗Kurz gefasst
Fachartikel in der Zeitschrift Proceedings of the AAAI Conference on Artificial Intelligence, online erschienen am 26.06.2023. Verfasst von Shiwei Li und 7 weiteren Autor:innen. Die Arbeit wurde bisher 20-mal zitiert (Crossref).
Abstract (Originalsprache)
Embedding tables are usually huge in click-through rate (CTR) prediction models. To train and deploy the CTR models efficiently and economically, it is necessary to compress their embedding tables. To this end, we formulate a novel quantization training paradigm to compress the embeddings from the training stage, termed low-precision training (LPT). Also, we provide theoretical analysis on its convergence. The results show that stochastic weight quantization has a faster convergence rate and a smaller convergence error than deterministic weight quantization in LPT. Further, to reduce accuracy degradation, we propose adaptive low-precision training (ALPT) which learns the step size (i.e., the quantization resolution). Experiments on two real-world datasets confirm our analysis and show that ALPT can significantly improve the prediction accuracy, especially at extremely low bit width. For the first time in CTR models, we successfully train 8-bit embeddings without sacrificing prediction accuracy.
Zitieren
APA
Li, S., Guo, H., Hou, L., Zhang, W., Tang, X., Tang, R., Zhang, R., & Li, R. (2023). Adaptive Low-Precision Training for Embeddings in Click-Through Rate Prediction. Proceedings of the AAAI Conference on Artificial Intelligence, 37(4), 4435-4443. https://doi.org/10.1609/aaai.v37i4.25564
Vancouver
Li S, Guo H, Hou L, Zhang W, Tang X, Tang R, et al. Adaptive Low-Precision Training for Embeddings in Click-Through Rate Prediction. Proceedings of the AAAI Conference on Artificial Intelligence. 2023;37(4):4435-4443. doi:10.1609/aaai.v37i4.25564
BibTeX
@article{Li2023,
author = {Li, Shiwei and Guo, Huifeng and Hou, Lu and Zhang, Wei and Tang, Xing and Tang, Ruiming and Zhang, Rui and Li, Ruixuan},
title = {Adaptive Low-Precision Training for Embeddings in Click-Through Rate Prediction},
journal = {Proceedings of the AAAI Conference on Artificial Intelligence},
year = {2023},
volume = {37},
number = {4},
pages = {4435--4443},
doi = {10.1609/aaai.v37i4.25564}
}
EUVIMED