EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

Temporal Consistency Transformer for Long Sequence Modeling

Anna Müller, Lukas Schneider, Clara Hoffmann

Journal of the Artificial Intelligence Engineers Consortium · 2026

DOI: 10.66238/jaiec69 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Journal of the Artificial Intelligence Engineers Consortium, 2026. Verfasst von Anna Müller und 2 weiteren Autor:innen. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Long-range dependency modeling is still difficult in sequence learning because useful information may weaken as the sequence becomes longer. This problem affects both language modeling and time-series prediction. This study proposes a sequence modeling method that combines Transformer learning, temporal consistency constraints, and a memory module. The temporal consistency term reduces sharp changes between adjacent hidden states. The memory module keeps useful historical information for later sequence steps. Experiments were conducted on WikiText-103 and a traffic forecasting dataset. The results showed that the proposed method reduced perplexity by about 11.5% in language modeling and reduced MAE by about 9.3% in traffic prediction. When the sequence length was greater than 512, overall performance improved by about 14%. These results show that temporal smoothing and memory updating can reduce information loss and improve long-sequence modeling. The method provides a practical approach for language modeling, traffic prediction, financial sequence analysis, and sensor monitoring.

Zitieren

APA

Müller, A., Schneider, L., & Hoffmann, C. (2026). Temporal Consistency Transformer for Long Sequence Modeling. Journal of the Artificial Intelligence Engineers Consortium. https://doi.org/10.66238/jaiec69

Vancouver

Müller A, Schneider L, Hoffmann C. Temporal Consistency Transformer for Long Sequence Modeling. Journal of the Artificial Intelligence Engineers Consortium. 2026. doi:10.66238/jaiec69

BibTeX
@article{Mller2026,
  author = {Müller, Anna and Schneider, Lukas and Hoffmann, Clara},
  title = {Temporal Consistency Transformer for Long Sequence Modeling},
  journal = {Journal of the Artificial Intelligence Engineers Consortium},
  year = {2026},
  doi = {10.66238/jaiec69}
}
Kostenlos starten →