Incorporating Uncertainty Quantification for the Performance Improvement of Academic Recommenders
Knowledge · 2023-06-27 · 3 (3) · 293-306
DOI: 10.3390/knowledge3030020 ↗At a glance
Journal article in Knowledge, published online on 2023-06-27. Written by Jie Zhu and 2 co-authors. Cited 1 times so far (Crossref); 41 references. The full text is freely available (open access, CC BY 4.0).
Abstract (original language)
Deep learning is widely used in many real-life applications. Despite their remarkable performance accuracies, deep learning networks are often poorly calibrated, which could be harmful in risk-sensitive scenarios. Uncertainty quantification offers a way to evaluate the reliability and trustworthiness of deep-learning-based model predictions. In this work, we introduced uncertainty quantification to our virtual research assistant recommender platform through both Monte Carlo dropout ensemble techniques. We also proposed a new formula to incorporate the uncertainty estimates into our recommendation models. The experiments were carried out on two different components of the recommender platform (i.e., a BERT-based grant recommender and a temporal graph network (TGN)-based collaborator recommender) using real-life datasets. The recommendation results were compared in terms of both recommender metrics (AUC, AP, etc.) and the calibration/reliability metric (ECE). With uncertainty quantification, we were able to better understand the behavior of our regular recommender outputs; while our BERT-based grant recommender tends to be overconfident with its outputs, our TGN-based collaborator recommender tends to be underconfident in producing matching probabilities. Initial case studies also showed that our proposed model with uncertainty quantification adjustment from ensemble gave the best-calibrated results together with the desirable recommender performance.
Cite
APA
Zhu, J., Novelo, L. L., & Yaseen, A. (2023). Incorporating Uncertainty Quantification for the Performance Improvement of Academic Recommenders. Knowledge, 3(3), 293-306. https://doi.org/10.3390/knowledge3030020
Vancouver
Zhu J, Novelo LL, Yaseen A. Incorporating Uncertainty Quantification for the Performance Improvement of Academic Recommenders. Knowledge. 2023;3(3):293-306. doi:10.3390/knowledge3030020
BibTeX
@article{Zhu2023,
author = {Zhu, Jie and Novelo, Luis Leon and Yaseen, Ashraf},
title = {Incorporating Uncertainty Quantification for the Performance Improvement of Academic Recommenders},
journal = {Knowledge},
year = {2023},
volume = {3},
number = {3},
pages = {293--306},
doi = {10.3390/knowledge3030020}
}
EUVIMED