Fidelity-Constrained Rule-Based Knowledge Distillation of Random Forests for Compact Breast Cancer Grade Classification

Vollständiger Abstract

Worum geht es in dieser Arbeit?

Breast cancer is a highly heterogeneous disease at the histopathological and molecular levels, and histological grade is a key prognostic indicator reflecting tumor biology. Although Random Forest models effectively capture nonlinear patterns in high-dimensional gene expression data, the decision logic of large tree ensembles remains difficult to inspect directly. This study proposes a knowledge-distillation framework that transfers the probabilistic decision behavior of a Random Forest teacher model to a compact, rule-based student model for distinguishing low- from high-histological-grade breast tumors using 20,385 gene-expression features from 1892 METABRIC samples, with clinical information used for sample matching and histological-grade label definition. Root-to-leaf decision paths extracted from the teacher were converted into a binary rule-activation matrix, followed by proximal group sparsification and fidelity-constrained Top-K rule selection. For the principal 400-tree teacher configuration, the student retained 65.0 ± 13.7 rules and achieved an ROC-AUC of 0.833 ± 0.023 and an average precision of 0.843 ± 0.019, while maintaining a raw teacher–student Pearson fidelity of 0.9607 ± 0.0112. As teacher capacity increased, the number of active decision paths increased substantially, whereas the selected rule budget remained within a comparatively narrow range and increased only modestly. The compact student also showed approximately 46% lower measured end-to-end inference latency and a 76% smaller serialized deployment size than the 400-tree teacher. Within the present METABRIC evaluation, these findings indicate that the structural complexity of a tree ensemble and the complexity required to approximate its probabilistic behavior need not scale proportionally, and that the proposed framework can provide a favorable trade-off among predictive performance, teacher fidelity, and deployment compactness.