CorrEvator: Evaluating Patch Correctness in Automated Program Repair From Online Community Discussions
IEEE Transactions on Consumer Electronics · 2026-05 · 72 (2) · 3539-3556
DOI: 10.1109/tce.2026.3676742 ↗Kurz gefasst
Fachartikel in der Zeitschrift IEEE Transactions on Consumer Electronics, 2026-05. Verfasst von Yuyang Wang und 8 weiteren Autor:innen. Die Arbeit nennt 84 Literaturangaben. Gefördert von: National Natural Science Foundation of China, Dalian Science and technology Innovation Fund Project, Fundamental Research Funds for the Central Universities.
Abstract (Originalsprache)
Automatic Program Repair (APR) aims to automatically fix software bugs by diagnosing the root cause of failures and suggesting potential solutions, effectively reducing developers’ debugging time and the cost and effort associated with bug identification and correction. However, existing works suffer from two challenges to efficient evaluation of the correctness of the patches: disregarding syntactic structure information in bug reports and patch descriptions, and lack of joint reasoning between bug reports and patch descriptions. These two challenges can hinder the ability of existing works to accurately discern subtle semantic differences between bug reports and patches, potentially resulting in incorrect evaluations. To address these challenges, this study proposes CorrEvator (Correct Patches Evaluator), a novel approach for evaluating patch correctness in APR. Specifically, CorrEvator incorporates the graph embedding component and the patch correctness evaluation component. The graph embedding component utilizes feature graphs to depict bug reports and patch descriptions, thereby encoding the syntactic structure and semantics of text elements such as titles and descriptions. This component effectively addresses the issue of disregarding syntactic structure information. Meanwhile, the patch correctness evaluation component facilitates information exchange and matching between different elements of the two feature graphs through an attention coefficient mechanism, thus resolving the challenge of lacking joint reasoning between peers. Experimental results on a large processed bug benchmark of 9,135 patches generated from three bug benchmarks (i.e., Defects4j, bug.jar and Bears) indicate that CorrEvator outperforms the state-of-the-art methods across various metrics including AUC, F1-score, and positive and negative recall. Compared with the state-of-the-art methods, CorrEvator’s AUC in predicting patch correctness is higher by 0.9%, 14.9%, 17.6%, 27.8%, 22.5% and 25.5%, respectively. The F1-score values are improved by 0.6%, 16.4%, 18.5%, 24%, 27%, and 11.2%, maintaining a recall rate of 93% for correct patches and filtering out 66.8% of incorrect ones. CorrEvator also achieves an average Accuracy of 85.2%.
Förderung
- National Natural Science Foundation of China
- Dalian Science and technology Innovation Fund Project
- Fundamental Research Funds for the Central Universities
Zitieren
APA
Wang, Y., Guo, S., Tao, Z., Li, H., Tan, J., Hao, L., Zhu, F., Chen, X., & Youssef, M. (2026). CorrEvator: Evaluating Patch Correctness in Automated Program Repair From Online Community Discussions. IEEE Transactions on Consumer Electronics, 72(2), 3539-3556. https://doi.org/10.1109/tce.2026.3676742
Vancouver
Wang Y, Guo S, Tao Z, Li H, Tan J, Hao L, et al. CorrEvator: Evaluating Patch Correctness in Automated Program Repair From Online Community Discussions. IEEE Transactions on Consumer Electronics. 2026;72(2):3539-3556. doi:10.1109/tce.2026.3676742
BibTeX
@article{Wang2026,
author = {Wang, Yuyang and Guo, Shikai and Tao, Zijian and Li, Hui and Tan, Jiaxi and Hao, Liying and Zhu, Fa and Chen, Xingchi and Youssef, Moustafa},
title = {CorrEvator: Evaluating Patch Correctness in Automated Program Repair From Online Community Discussions},
journal = {IEEE Transactions on Consumer Electronics},
year = {2026},
volume = {72},
number = {2},
pages = {3539--3556},
doi = {10.1109/tce.2026.3676742}
}
EUVIMED