EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

Total Correlation Information as a Sensitive Measure for Detecting Over-Smoothing in Graph Neural Networks

Amirahmad Nayyeri, Mohammad Taheri, Eghbal G. Mansoori

IEEE Access · 2026 · 14 · 67187-67203

DOI: 10.1109/access.2026.3685978 ↗

Kurz gefasst

Fachartikel in der Zeitschrift IEEE Access, 2026. Verfasst von Amirahmad Nayyeri und 2 weiteren Autor:innen. Die Arbeit nennt 76 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Graph Neural Networks (GNNs) achieve robust node representations by iteratively propagating information, a process that enforces a critical trade-off: deep message passing, essential for long-range dependency capture, ultimately causes the annihilation of short-range discriminative features. This leads to the detrimental over-smoothing problem, where node representations become globally indistinguishable. Existing solutions primarily rely on performance-based early stopping or architectural heuristics (e.g., residual connections), that fail to provide an intrinsic measure of information quality. This paper introduces a quantifiable information-theoretic framework to analyze the internal dynamics of GNNs. Our methodology maps the topological evolution of node representations onto a global binary sequence derived from Markovian diffusion on the network, allowing us to compute the Total Correlation Information (TCI) as a measure of structural compression. We demonstrate that the maximum point of this metric, ${\mathrm {TCI}}_{max}$ , serves as a universal optimal stopping criterion. Expanding upon traditional geometric metrics, a rigorous statistical examination reveals that the variance of maximal TCI ( $\sigma ^{2}\approx 0.0041$ ) is nearly ten times higher than the variance of the Mean Average Distance (MAD) ( $\sigma ^{2}\approx 0.00045$ ) at the same epoch, establishing TCI as a significantly more sensitive diagnostic instrument. Comparative analysis across different aggregation functions proves that the Graph Attention Network (GAT) achieves structural compression with superior parsimony, reaching optimal resolution in 8 epochs compared to 21 epochs for Mean aggregation at $\mu =0.3$ . In high-noise regimes ( $\mu =0.8$ ), where traditional aggregators collapse to a Distance $ \ge 2.4$ , GAT successfully reconstructs signals (Hamming Distance $ \le 0.25$ ) by maintaining a higher information-to-noise ratio through topological filtering. These results, validated against recent neighborhood-adaptive and domain-adaptive benchmarks, demonstrate that TCI provides a fundamental new perspective on GNN optimization and architectural efficiency.

Zitieren

APA

Nayyeri, A., Taheri, M., & Mansoori, E. G. (2026). Total Correlation Information as a Sensitive Measure for Detecting Over-Smoothing in Graph Neural Networks. IEEE Access, 14, 67187-67203. https://doi.org/10.1109/access.2026.3685978

Vancouver

Nayyeri A, Taheri M, Mansoori EG. Total Correlation Information as a Sensitive Measure for Detecting Over-Smoothing in Graph Neural Networks. IEEE Access. 2026;14:67187-67203. doi:10.1109/access.2026.3685978

BibTeX
@article{Nayyeri2026,
  author = {Nayyeri, Amirahmad and Taheri, Mohammad and Mansoori, Eghbal G.},
  title = {Total Correlation Information as a Sensitive Measure for Detecting Over-Smoothing in Graph Neural Networks},
  journal = {IEEE Access},
  year = {2026},
  volume = {14},
  pages = {67187--67203},
  doi = {10.1109/access.2026.3685978}
}
Kostenlos starten →