EUVIMEDEUVIMED entdecken
Crossref

Heat-kernel context compression for transformer representations using graph diffusion

Shashank Reddy Vadyala, Sai Nethra Betgeri

Academia AI and Applications · 06.11.2026 · 2 (4)

DOI: 10.20935/acadai8620 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Academia AI and Applications, online erschienen am 06.11.2026. Verfasst von Shashank Reddy Vadyala und 1 weiteren Autor:innen. Die Arbeit nennt 16 Literaturangaben.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Introduction

Long contexts often contain geometrically redundant internal representations. Heat Kernel Context Compression (HKCC) is a training-free representation–compression method for transformer systems in which internal embeddings and attention logits can be modified. Materials and methods: Context representations are placed on a sparse similarity graph. The method uses the random-walk Laplacian throughout, while symmetric normalization appears only through an explicit similarity transform used for stable Chebyshev computation. Heat residuals select anchors, and omitted representations are merged into multiplicity-weighted representatives.

Results

Corrected simulations with 340 unit-normalized vectors show that the random-walk heat kernel preserves row sums exactly and agrees with its similarity-transformed symmetric implementation to numerical precision. Across six controlled contexts, multiplicity-aware merging preserved approximately 0.99999 attention output cosine fidelity at 8% and 12% retention, whereas residual-only pruning degraded sharply. Anchor rules were effectively tied under this synthetic regime.

Conclusions

HKCC is supported as a controlled proof of concept for representation–compression in self-hosted or otherwise modifiable transformer stacks. The current study does not establish savings for commercial text application programming interfaces (APIs), downstream task preservation, or universal superiority of heat residual anchors. The added utility of diffusion, real-model key/value and positional integration, and end-to-end prefill speedups remain unestablished.

Zitieren

APA

Vadyala, S. R., & Betgeri, S. N. (2026). Heat-kernel context compression for transformer representations using graph diffusion. Academia AI and Applications, 2(4). https://doi.org/10.20935/acadai8620

Vancouver

Vadyala SR, Betgeri SN. Heat-kernel context compression for transformer representations using graph diffusion. Academia AI and Applications. 2026;2(4). doi:10.20935/acadai8620

BibTeX
@article{Vadyala2026,
  author = {Vadyala, Shashank Reddy and Betgeri, Sai Nethra},
  title = {Heat-kernel context compression for transformer representations using graph diffusion},
  journal = {Academia AI and Applications},
  year = {2026},
  volume = {2},
  number = {4},
  doi = {10.20935/acadai8620}
}
Kostenlos starten →