EUVIMEDEUVIMED entdecken
Crossref

JELV: A Judge of Edit-Level Validity for Evaluation and Automated Reference Expansion in Grammatical Error Correction

Yuhao Zhan, Yuqing Zhang, Jing Yuan, Qixiang Ma, Zhiqi Yang, Yu Gu, Zemin Liu, Fei Wu

Proceedings of the AAAI Conference on Artificial Intelligence · 14.03.2026 · 40 (41) · 34611-34619

DOI: 10.1609/aaai.v40i41.40761 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Proceedings of the AAAI Conference on Artificial Intelligence, online erschienen am 14.03.2026. Verfasst von Yuhao Zhan und 7 weiteren Autor:innen.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Existing Grammatical Error Correction (GEC) systems suffer from limited reference diversity, leading to underestimated evaluation and restricted model generalization. To address this issue, we introduce the Judge of Edit-Level Validity (JELV), an automated framework to validate correction edits from grammaticality, faithfulness, and fluency. Using our proposed human-annotated Pair-wise Edit-level Validity Dataset (PEVData) as benchmark, JELV offers two implementations: a multi-turn LLM-as-Judges pipeline achieving 90% agreement with human annotators, and a distilled DeBERTa classifier with 85% precision on valid edits. We then apply JELV to reclassify misjudged false positives in evaluation and derive a comprehensive evaluation metric by integrating false positive decoupling and fluency scoring, resulting in state-of-the-art correlation with human judgments. We also apply JELV to filter LLM-generated correction candidates, expanding the BEA19's single-reference dataset containing 38,692 source sentences. Retraining top GEC systems on this expanded dataset yields measurable performance gains. JELV provides a scalable solution for enhancing reference diversity and strengthening both evaluation and model generalization.

Zitieren

APA

Zhan, Y., Zhang, Y., Yuan, J., Ma, Q., Yang, Z., Gu, Y., Liu, Z., & Wu, F. (2026). JELV: A Judge of Edit-Level Validity for Evaluation and Automated Reference Expansion in Grammatical Error Correction. Proceedings of the AAAI Conference on Artificial Intelligence, 40(41), 34611-34619. https://doi.org/10.1609/aaai.v40i41.40761

Vancouver

Zhan Y, Zhang Y, Yuan J, Ma Q, Yang Z, Gu Y, et al. JELV: A Judge of Edit-Level Validity for Evaluation and Automated Reference Expansion in Grammatical Error Correction. Proceedings of the AAAI Conference on Artificial Intelligence. 2026;40(41):34611-34619. doi:10.1609/aaai.v40i41.40761

BibTeX
@article{Zhan2026,
  author = {Zhan, Yuhao and Zhang, Yuqing and Yuan, Jing and Ma, Qixiang and Yang, Zhiqi and Gu, Yu and Liu, Zemin and Wu, Fei},
  title = {JELV: A Judge of Edit-Level Validity for Evaluation and Automated Reference Expansion in Grammatical Error Correction},
  journal = {Proceedings of the AAAI Conference on Artificial Intelligence},
  year = {2026},
  volume = {40},
  number = {41},
  pages = {34611--34619},
  doi = {10.1609/aaai.v40i41.40761}
}
Kostenlos starten →