Fractional Calculus for Large Language Models: A Survey of Potentials, Mechanisms, and Challenges
Fractal and Fractional · 03.07.2026 · 10 (7) · 454
DOI: 10.3390/fractalfract10070454 ↗Kurz gefasst
Fachartikel in der Zeitschrift Fractal and Fractional, online erschienen am 03.07.2026. Verfasst von Wenwen Tu und 4 weiteren Autor:innen. Die Arbeit wurde bisher 1-mal zitiert (Crossref) und nennt 111 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: Yunnan Provincial Department of Education, Kunming University of Science and Technology, The Ministry of Education.
Abstract (Originalsprache)
Large language models (LLMs) face critical bottlenecks like the “memory wall” and context degradation due to standard integer-order Markovian assumptions. This survey reviews integrating fractional calculus to explore potential solutions. Synthesizing the recent literature reveals a transitional landscape: while fractional mechanisms yield significant empirical gains in medium-scale architectures (e.g., up to 14.36% absolute accuracy improvements), explicit validation on autoregressive models exceeding one billion parameters remains largely unexplored. By introducing heavy-tailed power-law memory and non-local historical gradients, fractional operators help mitigate catastrophic forgetting and optimization instability. Furthermore, we propose dual-integration frameworks: a decoupled neuro-symbolic architecture for scientific computing, and an endogenous fractional evolution architecture for native long-context reasoning. While fractional dynamics offer mathematical pathways to alleviate LLM constraints, computational overhead and training stability challenges remain. Future research should prioritize selective fractionalization and data-driven scientific discovery.
Förderung
- Yunnan Provincial Department of Education
- Kunming University of Science and Technology
- The Ministry of Education
- Yunnan Province Science and Technology Department
Zitieren
APA
Tu, W., Liu, X., Wang, H., Li, X., & Han, C. (2026). Fractional Calculus for Large Language Models: A Survey of Potentials, Mechanisms, and Challenges. Fractal and Fractional, 10(7), 454. https://doi.org/10.3390/fractalfract10070454
Vancouver
Tu W, Liu X, Wang H, Li X, Han C. Fractional Calculus for Large Language Models: A Survey of Potentials, Mechanisms, and Challenges. Fractal and Fractional. 2026;10(7):454. doi:10.3390/fractalfract10070454
BibTeX
@article{Tu2026,
author = {Tu, Wenwen and Liu, Xixing and Wang, Hexin and Li, Xuhang and Han, Chunyang},
title = {Fractional Calculus for Large Language Models: A Survey of Potentials, Mechanisms, and Challenges},
journal = {Fractal and Fractional},
year = {2026},
volume = {10},
number = {7},
pages = {454},
doi = {10.3390/fractalfract10070454}
}
EUVIMED