Temporal Consistency Transformer for Long Sequence Modeling
Journal of the Artificial Intelligence Engineers Consortium · 2026
DOI: 10.66238/jaiec69 ↗Kurz gefasst
Fachartikel in der Zeitschrift Journal of the Artificial Intelligence Engineers Consortium, 2026. Verfasst von Anna Müller und 2 weiteren Autor:innen. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).
Abstract (Originalsprache)
Long-range dependency modeling is still difficult in sequence learning because useful information may weaken as the sequence becomes longer. This problem affects both language modeling and time-series prediction. This study proposes a sequence modeling method that combines Transformer learning, temporal consistency constraints, and a memory module. The temporal consistency term reduces sharp changes between adjacent hidden states. The memory module keeps useful historical information for later sequence steps. Experiments were conducted on WikiText-103 and a traffic forecasting dataset. The results showed that the proposed method reduced perplexity by about 11.5% in language modeling and reduced MAE by about 9.3% in traffic prediction. When the sequence length was greater than 512, overall performance improved by about 14%. These results show that temporal smoothing and memory updating can reduce information loss and improve long-sequence modeling. The method provides a practical approach for language modeling, traffic prediction, financial sequence analysis, and sensor monitoring.
Zitieren
APA
Müller, A., Schneider, L., & Hoffmann, C. (2026). Temporal Consistency Transformer for Long Sequence Modeling. Journal of the Artificial Intelligence Engineers Consortium. https://doi.org/10.66238/jaiec69
Vancouver
Müller A, Schneider L, Hoffmann C. Temporal Consistency Transformer for Long Sequence Modeling. Journal of the Artificial Intelligence Engineers Consortium. 2026. doi:10.66238/jaiec69
BibTeX
@article{Mller2026,
author = {Müller, Anna and Schneider, Lukas and Hoffmann, Clara},
title = {Temporal Consistency Transformer for Long Sequence Modeling},
journal = {Journal of the Artificial Intelligence Engineers Consortium},
year = {2026},
doi = {10.66238/jaiec69}
}
EUVIMED