EUVIMEDEUVIMED entdecken
Crossref

Adaptive Low-Precision Training for Embeddings in Click-Through Rate Prediction

Shiwei Li, Huifeng Guo, Lu Hou, Wei Zhang, Xing Tang, Ruiming Tang, Rui Zhang, Ruixuan Li

Proceedings of the AAAI Conference on Artificial Intelligence · 26.06.2023 · 37 (4) · 4435-4443

DOI: 10.1609/aaai.v37i4.25564 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Proceedings of the AAAI Conference on Artificial Intelligence, online erschienen am 26.06.2023. Verfasst von Shiwei Li und 7 weiteren Autor:innen. Die Arbeit wurde bisher 20-mal zitiert (Crossref).

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Embedding tables are usually huge in click-through rate (CTR) prediction models. To train and deploy the CTR models efficiently and economically, it is necessary to compress their embedding tables. To this end, we formulate a novel quantization training paradigm to compress the embeddings from the training stage, termed low-precision training (LPT). Also, we provide theoretical analysis on its convergence. The results show that stochastic weight quantization has a faster convergence rate and a smaller convergence error than deterministic weight quantization in LPT. Further, to reduce accuracy degradation, we propose adaptive low-precision training (ALPT) which learns the step size (i.e., the quantization resolution). Experiments on two real-world datasets confirm our analysis and show that ALPT can significantly improve the prediction accuracy, especially at extremely low bit width. For the first time in CTR models, we successfully train 8-bit embeddings without sacrificing prediction accuracy.

Zitieren

APA

Li, S., Guo, H., Hou, L., Zhang, W., Tang, X., Tang, R., Zhang, R., & Li, R. (2023). Adaptive Low-Precision Training for Embeddings in Click-Through Rate Prediction. Proceedings of the AAAI Conference on Artificial Intelligence, 37(4), 4435-4443. https://doi.org/10.1609/aaai.v37i4.25564

Vancouver

Li S, Guo H, Hou L, Zhang W, Tang X, Tang R, et al. Adaptive Low-Precision Training for Embeddings in Click-Through Rate Prediction. Proceedings of the AAAI Conference on Artificial Intelligence. 2023;37(4):4435-4443. doi:10.1609/aaai.v37i4.25564

BibTeX
@article{Li2023,
  author = {Li, Shiwei and Guo, Huifeng and Hou, Lu and Zhang, Wei and Tang, Xing and Tang, Ruiming and Zhang, Rui and Li, Ruixuan},
  title = {Adaptive Low-Precision Training for Embeddings in Click-Through Rate Prediction},
  journal = {Proceedings of the AAAI Conference on Artificial Intelligence},
  year = {2023},
  volume = {37},
  number = {4},
  pages = {4435--4443},
  doi = {10.1609/aaai.v37i4.25564}
}
Kostenlos starten →