Total Correlation Information as a Sensitive Measure for Detecting Over-Smoothing in Graph Neural Networks
IEEE Access · 2026 · 14 · 67187-67203
DOI: 10.1109/access.2026.3685978 ↗Kurz gefasst
Fachartikel in der Zeitschrift IEEE Access, 2026. Verfasst von Amirahmad Nayyeri und 2 weiteren Autor:innen. Die Arbeit nennt 76 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).
Abstract (Originalsprache)
Graph Neural Networks (GNNs) achieve robust node representations by iteratively propagating information, a process that enforces a critical trade-off: deep message passing, essential for long-range dependency capture, ultimately causes the annihilation of short-range discriminative features. This leads to the detrimental over-smoothing problem, where node representations become globally indistinguishable. Existing solutions primarily rely on performance-based early stopping or architectural heuristics (e.g., residual connections), that fail to provide an intrinsic measure of information quality. This paper introduces a quantifiable information-theoretic framework to analyze the internal dynamics of GNNs. Our methodology maps the topological evolution of node representations onto a global binary sequence derived from Markovian diffusion on the network, allowing us to compute the Total Correlation Information (TCI) as a measure of structural compression. We demonstrate that the maximum point of this metric, ${\mathrm {TCI}}_{max}$ , serves as a universal optimal stopping criterion. Expanding upon traditional geometric metrics, a rigorous statistical examination reveals that the variance of maximal TCI ( $\sigma ^{2}\approx 0.0041$ ) is nearly ten times higher than the variance of the Mean Average Distance (MAD) ( $\sigma ^{2}\approx 0.00045$ ) at the same epoch, establishing TCI as a significantly more sensitive diagnostic instrument. Comparative analysis across different aggregation functions proves that the Graph Attention Network (GAT) achieves structural compression with superior parsimony, reaching optimal resolution in 8 epochs compared to 21 epochs for Mean aggregation at $\mu =0.3$ . In high-noise regimes ( $\mu =0.8$ ), where traditional aggregators collapse to a Distance $ \ge 2.4$ , GAT successfully reconstructs signals (Hamming Distance $ \le 0.25$ ) by maintaining a higher information-to-noise ratio through topological filtering. These results, validated against recent neighborhood-adaptive and domain-adaptive benchmarks, demonstrate that TCI provides a fundamental new perspective on GNN optimization and architectural efficiency.
Zitieren
APA
Nayyeri, A., Taheri, M., & Mansoori, E. G. (2026). Total Correlation Information as a Sensitive Measure for Detecting Over-Smoothing in Graph Neural Networks. IEEE Access, 14, 67187-67203. https://doi.org/10.1109/access.2026.3685978
Vancouver
Nayyeri A, Taheri M, Mansoori EG. Total Correlation Information as a Sensitive Measure for Detecting Over-Smoothing in Graph Neural Networks. IEEE Access. 2026;14:67187-67203. doi:10.1109/access.2026.3685978
BibTeX
@article{Nayyeri2026,
author = {Nayyeri, Amirahmad and Taheri, Mohammad and Mansoori, Eghbal G.},
title = {Total Correlation Information as a Sensitive Measure for Detecting Over-Smoothing in Graph Neural Networks},
journal = {IEEE Access},
year = {2026},
volume = {14},
pages = {67187--67203},
doi = {10.1109/access.2026.3685978}
}
EUVIMED