EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

A Multi-Modal Retrieval Model for Mathematical Expressions Based on ConvNeXt and Hesitant Fuzzy Set

Ruxuan Li, Jingyi Wang, Xuedong Tian

Electronics · 20.10.2023 · 12 (20) · 4363

DOI: 10.3390/electronics12204363 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Electronics, online erschienen am 20.10.2023. Verfasst von Ruxuan Li und 2 weiteren Autor:innen. Die Arbeit wurde bisher 4-mal zitiert (Crossref) und nennt 34 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: Natural Science Foundation of Hebei Province of China.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Mathematical expression retrieval is an essential component of mathematical information retrieval. Current mathematical expression retrieval research primarily targets single modalities, particularly text, which can lead to the loss of structural information. On the other hand, multimodal research has demonstrated promising outcomes across different domains, and mathematical expressions in image format are adept at preserving their structural characteristics. So we propose a multi-modal retrieval model for mathematical expressions based on ConvNeXt and HFS to address the limitations of single-modal retrieval. For the image modal, mathematical expression retrieval is based on the similarity of image features and symbol-level features of the expression, where image features of the expression image are extracted by ConvNeXt, while symbol-level features are obtained by the Symbol Level Features Extraction (SLFE) module. For the text modal, the Formula Description Structure (FDS) is employed to analyze expressions and extract their attributes. Additionally, the application of the Hesitant Fuzzy Set (HFS) theory facilitates the computation of hesitant fuzzy similarity between mathematical queries and candidate expressions. Finally, Reciprocal Rank Fusion (RRF) is employed to integrate rankings from image modal and text modal retrieval, yielding the ultimate retrieval list. The experiment was conducted on the publicly accessible ArXiv dataset (containing 592,345 mathematical expressions) and the NTCIR-mair-wikipedia-corpus (NTCIR) dataset.The MAP@10 values for the multimodal RRF fusion approach are recorded as 0.774. These substantiate the efficacy of the multi-modal mathematical expression retrieval approach based on ConvNeXt and HFS.

Förderung

  • Natural Science Foundation of Hebei Province of China

Zitieren

APA

Li, R., Wang, J., & Tian, X. (2023). A Multi-Modal Retrieval Model for Mathematical Expressions Based on ConvNeXt and Hesitant Fuzzy Set. Electronics, 12(20), 4363. https://doi.org/10.3390/electronics12204363

Vancouver

Li R, Wang J, Tian X. A Multi-Modal Retrieval Model for Mathematical Expressions Based on ConvNeXt and Hesitant Fuzzy Set. Electronics. 2023;12(20):4363. doi:10.3390/electronics12204363

BibTeX
@article{Li2023,
  author = {Li, Ruxuan and Wang, Jingyi and Tian, Xuedong},
  title = {A Multi-Modal Retrieval Model for Mathematical Expressions Based on ConvNeXt and Hesitant Fuzzy Set},
  journal = {Electronics},
  year = {2023},
  volume = {12},
  number = {20},
  pages = {4363},
  doi = {10.3390/electronics12204363}
}
Kostenlos starten →