EUVIMEDEUVIMED entdecken
Crossref

Enhancing temporal commonsense understanding using disentangled attention-based method with a hybrid data framework

Wasif Feroze, Muhammad Shahid, Shaohuan Cheng, Elias Lemuye Jimale, Yi Yang, Hong Qu, Yulin Wang

Intelligence & Robotics · 19.03.2025 · 5 (1) · 228-47

DOI: 10.20517/ir.2025.12 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Intelligence & Robotics, online erschienen am 19.03.2025. Verfasst von Wasif Feroze und 6 weiteren Autor:innen. Die Arbeit wurde bisher 5-mal zitiert (Crossref) und nennt 59 Literaturangaben.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Understanding and capturing temporal relationships between time-related events expressed in text is a crucial aspect of natural language understanding (NLU). Although transformer-based pre-trained language models such as bidirectional encoder representations from transformers (BERT) have achieved significant success in various natural language processing (NLP) tasks, they are still believed to underperform in temporal commonsense tasks due to the limitation of vanilla self-attention. This paper proposes a methodology for developing language models to understand temporal commonsense reasoning over several tasks better. The proposed framework integrates a multi-data hybrid curation approach for dataset preparation, a collaborative synthetic dataset generation process involving chat agents and human domain experts, and a multi-stage fine-tuning strategy that leverages curated, intermediate, and target datasets to enhance temporal commonsense reasoning capabilities. The models we use in our proposed methodology are superior due to the use of an advanced attention mechanism and effective utilization of our framework. These models utilize disentangled attention, which is relative encoding position, which proved crucial for temporal commonsense by understanding temporal cues and indicators efficiently. Our extensive experiments show that models built with our proposed methodology enhance results on several temporal commonsense categories. Our results show that we achieved better performance than the previous published work by utilizing a disentangled attention mechanism and hybrid data framework. Most impressively, our approach has demonstrated state-of-the-art (SOTA) results, surpassing all previous studies on temporal commonsense for the MC-TACO dataset.

Zitieren

APA

Feroze, W., Shahid, M., Cheng, S., Jimale, E. L., Yang, Y., Qu, H., & Wang, Y. (2025). Enhancing temporal commonsense understanding using disentangled attention-based method with a hybrid data framework. Intelligence & Robotics, 5(1), 228-47. https://doi.org/10.20517/ir.2025.12

Vancouver

Feroze W, Shahid M, Cheng S, Jimale EL, Yang Y, Qu H, et al. Enhancing temporal commonsense understanding using disentangled attention-based method with a hybrid data framework. Intelligence & Robotics. 2025;5(1):228-47. doi:10.20517/ir.2025.12

BibTeX
@article{Feroze2025,
  author = {Feroze, Wasif and Shahid, Muhammad and Cheng, Shaohuan and Jimale, Elias Lemuye and Yang, Yi and Qu, Hong and Wang, Yulin},
  title = {Enhancing temporal commonsense understanding using disentangled attention-based method with a hybrid data framework},
  journal = {Intelligence & Robotics},
  year = {2025},
  volume = {5},
  number = {1},
  pages = {228--47},
  doi = {10.20517/ir.2025.12}
}
Kostenlos starten →