EUVIMEDEUVIMED entdecken
Crossref

Scaling Law for Recommendation Models: Towards General-Purpose User Representations

Kyuyong Shin, Hanock Kwak, Su Young Kim, Max Nihlén Ramström, Jisu Jeong, Jung-Woo Ha, Kyung-Min Kim

Proceedings of the AAAI Conference on Artificial Intelligence · 26.06.2023 · 37 (4) · 4596-4604

DOI: 10.1609/aaai.v37i4.25582 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Proceedings of the AAAI Conference on Artificial Intelligence, online erschienen am 26.06.2023. Verfasst von Kyuyong Shin und 6 weiteren Autor:innen. Die Arbeit wurde bisher 25-mal zitiert (Crossref).

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Recent advancement of large-scale pretrained models such as BERT, GPT-3, CLIP, and Gopher, has shown astonishing achievements across various task domains. Unlike vision recognition and language models, studies on general-purpose user representation at scale still remain underexplored. Here we explore the possibility of general-purpose user representation learning by training a universal user encoder at large scales. We demonstrate that the scaling law is present in user representation learning areas, where the training error scales as a power-law with the amount of computation. Our Contrastive Learning User Encoder (CLUE), optimizes task-agnostic objectives, and the resulting user embeddings stretch our expectation of what is possible to do in various downstream tasks. CLUE also shows great transferability to other domains and companies, as performances on an online experiment shows significant improvements in Click-Through-Rate (CTR). Furthermore, we also investigate how the model performance is influenced by the scale factors, such as training data size, model capacity, sequence length, and batch size. Finally, we discuss the broader impacts of CLUE in general.

Zitieren

APA

Shin, K., Kwak, H., Kim, S. Y., Ramström, M. N., Jeong, J., Ha, J., & Kim, K. (2023). Scaling Law for Recommendation Models: Towards General-Purpose User Representations. Proceedings of the AAAI Conference on Artificial Intelligence, 37(4), 4596-4604. https://doi.org/10.1609/aaai.v37i4.25582

Vancouver

Shin K, Kwak H, Kim SY, Ramström MN, Jeong J, Ha J, et al. Scaling Law for Recommendation Models: Towards General-Purpose User Representations. Proceedings of the AAAI Conference on Artificial Intelligence. 2023;37(4):4596-4604. doi:10.1609/aaai.v37i4.25582

BibTeX
@article{Shin2023,
  author = {Shin, Kyuyong and Kwak, Hanock and Kim, Su Young and Ramström, Max Nihlén and Jeong, Jisu and Ha, Jung-Woo and Kim, Kyung-Min},
  title = {Scaling Law for Recommendation Models: Towards General-Purpose User Representations},
  journal = {Proceedings of the AAAI Conference on Artificial Intelligence},
  year = {2023},
  volume = {37},
  number = {4},
  pages = {4596--4604},
  doi = {10.1609/aaai.v37i4.25582}
}
Kostenlos starten →