EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

Retrieval-style In-context Learning for Few-shot Hierarchical Text Classification

Huiyao Chen, Yu Zhao, Zulong Chen, Mengjia Wang, Liangyue Li, Meishan Zhang, Min Zhang

Transactions of the Association for Computational Linguistics · 30.09.2024 · 12 · 1214-1231

DOI: 10.1162/tacl_a_00697 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Transactions of the Association for Computational Linguistics, online erschienen am 30.09.2024. Verfasst von Huiyao Chen und 6 weiteren Autor:innen. Die Arbeit wurde bisher 25-mal zitiert (Crossref) und nennt 55 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Hierarchical text classification (HTC) is an important task with broad applications, and few-shot HTC has gained increasing interest recently. While in-context learning (ICL) with large language models (LLMs) has achieved significant success in few-shot learning, it is not as effective for HTC because of the expansive hierarchical label sets and extremely ambiguous labels. In this work, we introduce the first ICL-based framework with LLM for few-shot HTC. We exploit a retrieval database to identify relevant demonstrations, and an iterative policy to manage multi-layer hierarchical labels. Particularly, we equip the retrieval database with HTC label-aware representations for the input texts, which is achieved by continual training on a pretrained language model with masked language modeling (MLM), layer-wise classification (CLS, specifically for HTC), and a novel divergent contrastive learning (DCL, mainly for adjacent semantically similar labels) objective. Experimental results on three benchmark datasets demonstrate superior performance of our method, and we can achieve state-of-the-art results in few-shot HTC.

Zitieren

APA

Chen, H., Zhao, Y., Chen, Z., Wang, M., Li, L., Zhang, M., & Zhang, M. (2024). Retrieval-style In-context Learning for Few-shot Hierarchical Text Classification. Transactions of the Association for Computational Linguistics, 12, 1214-1231. https://doi.org/10.1162/tacl_a_00697

Vancouver

Chen H, Zhao Y, Chen Z, Wang M, Li L, Zhang M, et al. Retrieval-style In-context Learning for Few-shot Hierarchical Text Classification. Transactions of the Association for Computational Linguistics. 2024;12:1214-1231. doi:10.1162/tacl_a_00697

BibTeX
@article{Chen2024,
  author = {Chen, Huiyao and Zhao, Yu and Chen, Zulong and Wang, Mengjia and Li, Liangyue and Zhang, Meishan and Zhang, Min},
  title = {Retrieval-style In-context Learning for Few-shot Hierarchical Text Classification},
  journal = {Transactions of the Association for Computational Linguistics},
  year = {2024},
  volume = {12},
  pages = {1214--1231},
  doi = {10.1162/tacl_a_00697}
}
Kostenlos starten →