EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

Anisotropic span embeddings and the negative impact of higher-order inference for coreference resolution: An empirical analysis

Feng Hou, Ruili Wang, See-Kiong Ng, Fangyi Zhu, Michael Witbrock, Steven F. Cahan, Lily Chen, Xiaoyun Jia

Natural Language Engineering · 2024-11 · 30 (6) · 1301-1322

DOI: 10.1017/s1351324924000019 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Natural Language Engineering, online erschienen am 25.01.2024. Verfasst von Feng Hou und 7 weiteren Autor:innen. Die Arbeit wurde bisher 5-mal zitiert (Crossref) und nennt 40 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Coreference resolution is the task of identifying and clustering mentions that refer to the same entity in a document. Based on state-of-the-art deep learning approaches, end-to-end coreference resolution considers all spans as candidate mentions and tackles mention detection and coreference resolution simultaneously. Recently, researchers have attempted to incorporate document-level context using higher-order inference (HOI) to improve end-to-end coreference resolution. However, HOI methods have been shown to have marginal or even negative impact on coreference resolution. In this paper, we reveal the reasons for the negative impact of HOI coreference resolution. Contextualized representations (e.g., those produced by BERT) for building span embeddings have been shown to be highly anisotropic. We show that HOI actually increases and thus worsens the anisotropy of span embeddings and makes it difficult to distinguish between related but distinct entities (e.g., pilots and flight attendants ). Instead of using HOI, we propose two methods, Less-Anisotropic Internal Representations (LAIR) and Data Augmentation with Document Synthesis and Mention Swap (DSMS), to learn less-anisotropic span embeddings for coreference resolution. LAIR uses a linear aggregation of the first layer and the topmost layer of contextualized embeddings. DSMS generates more diversified examples of related but distinct entities by synthesizing documents and by mention swapping. Our experiments show that less-anisotropic span embeddings improve the performance significantly (+2.8 F1 gain on the OntoNotes benchmark) reaching new state-of-the-art performance on the GAP dataset.

Zitieren

APA

Hou, F., Wang, R., Ng, S., Zhu, F., Witbrock, M., Cahan, S. F., Chen, L., & Jia, X. (2024). Anisotropic span embeddings and the negative impact of higher-order inference for coreference resolution: An empirical analysis. Natural Language Engineering, 30(6), 1301-1322. https://doi.org/10.1017/s1351324924000019

Vancouver

Hou F, Wang R, Ng S, Zhu F, Witbrock M, Cahan SF, et al. Anisotropic span embeddings and the negative impact of higher-order inference for coreference resolution: An empirical analysis. Natural Language Engineering. 2024;30(6):1301-1322. doi:10.1017/s1351324924000019

BibTeX
@article{Hou2024,
  author = {Hou, Feng and Wang, Ruili and Ng, See-Kiong and Zhu, Fangyi and Witbrock, Michael and Cahan, Steven F. and Chen, Lily and Jia, Xiaoyun},
  title = {Anisotropic span embeddings and the negative impact of higher-order inference for coreference resolution: An empirical analysis},
  journal = {Natural Language Engineering},
  year = {2024},
  volume = {30},
  number = {6},
  pages = {1301--1322},
  doi = {10.1017/s1351324924000019}
}
Kostenlos starten →