Incorporating Uncertainty Quantification for the Performance Improvement of Academic Recommenders
Knowledge · 27.06.2023 · 3 (3) · 293-306
DOI: 10.3390/knowledge3030020 ↗Kurz gefasst
Fachartikel in der Zeitschrift Knowledge, online erschienen am 27.06.2023. Verfasst von Jie Zhu und 2 weiteren Autor:innen. Die Arbeit wurde bisher 1-mal zitiert (Crossref) und nennt 41 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).
Abstract (Originalsprache)
Deep learning is widely used in many real-life applications. Despite their remarkable performance accuracies, deep learning networks are often poorly calibrated, which could be harmful in risk-sensitive scenarios. Uncertainty quantification offers a way to evaluate the reliability and trustworthiness of deep-learning-based model predictions. In this work, we introduced uncertainty quantification to our virtual research assistant recommender platform through both Monte Carlo dropout ensemble techniques. We also proposed a new formula to incorporate the uncertainty estimates into our recommendation models. The experiments were carried out on two different components of the recommender platform (i.e., a BERT-based grant recommender and a temporal graph network (TGN)-based collaborator recommender) using real-life datasets. The recommendation results were compared in terms of both recommender metrics (AUC, AP, etc.) and the calibration/reliability metric (ECE). With uncertainty quantification, we were able to better understand the behavior of our regular recommender outputs; while our BERT-based grant recommender tends to be overconfident with its outputs, our TGN-based collaborator recommender tends to be underconfident in producing matching probabilities. Initial case studies also showed that our proposed model with uncertainty quantification adjustment from ensemble gave the best-calibrated results together with the desirable recommender performance.
Zitieren
APA
Zhu, J., Novelo, L. L., & Yaseen, A. (2023). Incorporating Uncertainty Quantification for the Performance Improvement of Academic Recommenders. Knowledge, 3(3), 293-306. https://doi.org/10.3390/knowledge3030020
Vancouver
Zhu J, Novelo LL, Yaseen A. Incorporating Uncertainty Quantification for the Performance Improvement of Academic Recommenders. Knowledge. 2023;3(3):293-306. doi:10.3390/knowledge3030020
BibTeX
@article{Zhu2023,
author = {Zhu, Jie and Novelo, Luis Leon and Yaseen, Ashraf},
title = {Incorporating Uncertainty Quantification for the Performance Improvement of Academic Recommenders},
journal = {Knowledge},
year = {2023},
volume = {3},
number = {3},
pages = {293--306},
doi = {10.3390/knowledge3030020}
}
EUVIMED