EUVIMEDEUVIMED entdecken
Crossref

A Decade of Architectural Evolution: A Comprehensive Survey of Deep Learning Model Implementations

Md Ahnaf Tajwar Kamal, Nawfat Kamal Munifa, Fahmida Yasmin

American Journal of Data Science and Analytics · 01.10.2025 · 06 (10) · 32-42

DOI: 10.63125/y7e5vr53 ↗

Kurz gefasst

Fachartikel in der Zeitschrift American Journal of Data Science and Analytics, online erschienen am 01.10.2025. Verfasst von Md Ahnaf Tajwar Kamal und 2 weiteren Autor:innen.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

This comprehensive survey paper presents a rigorous, systematic evaluation of deep learning model implementations, engineering milestones, and cross-layer architectural transformations over the highly transformative decade spanning from 2016 to 2026. During this chronological period, the global artificial intelligence paradigm shifted decisively from highly heterogeneous, task-specific convolutional configurations and recurrent layer hierarchies to standardized, uniform, global Transformer-based execution pipelines, ultimately consolidating into contemporary sparse Mixture of Experts (MoE) topologies and multi-stage test-time compute reinforcement layouts. We map out the granular mathematical and systemic developments that enabled this massive scaling behavior, focusing heavily on core attention mechanism modifications (including Multi-Head, Multi-Query, Grouped-Query, and Multi-head Latent Attention structures), performance-driven layer normalization changes (such as LayerNorm and RMSNorm frameworks), and rotary positional transformations. Additionally, we analyze system-level hardware-aware engineering breakthroughs that successfully mitigated physical compute boundaries, highlighting critical memory tiling strategies like FlashAttention, sharded optimizer parameters via ZeRO optimization pipelines, and mixed-precision low-bit computation layouts (FP8/FP4 mixed tensor configurations). Finally, we characterize the latest 2025–2026 operational shift toward dynamic test-time scaling systems, where inference-phase processing budgets scale fluidly with automated verification chains, tree search methods, and self-correction loops. By structuring this comprehensive multi-layer technological taxonomy, this survey traces the critical structural, co-designed software-hardware execution pipelines that form the foundational baseline of modern generative artificial intelligence networks across global deployment landscapes.

Zitieren

APA

Kamal, M. A. T., Munifa, N. K., & Yasmin, F. (2025). A Decade of Architectural Evolution: A Comprehensive Survey of Deep Learning Model Implementations. American Journal of Data Science and Analytics, 06(10), 32-42. https://doi.org/10.63125/y7e5vr53

Vancouver

Kamal MAT, Munifa NK, Yasmin F. A Decade of Architectural Evolution: A Comprehensive Survey of Deep Learning Model Implementations. American Journal of Data Science and Analytics. 2025;06(10):32-42. doi:10.63125/y7e5vr53

BibTeX
@article{Kamal2025,
  author = {Kamal, Md Ahnaf Tajwar and Munifa, Nawfat Kamal and Yasmin, Fahmida},
  title = {A Decade of Architectural Evolution: A Comprehensive Survey of Deep Learning Model Implementations},
  journal = {American Journal of Data Science and Analytics},
  year = {2025},
  volume = {06},
  number = {10},
  pages = {32--42},
  doi = {10.63125/y7e5vr53}
}
Kostenlos starten →