Semantic Feature Clustering Using LLM Embeddings for Label-Dependent Feature Selection
International Journal of Advanced Computer Science and Applications · 2026 · 17 (10)
DOI: 10.14569/ijacsa.2026.0171049 ↗Kurz gefasst
Fachartikel in der Zeitschrift International Journal of Advanced Computer Science and Applications, 2026. Verfasst von Shikha Choudhary und 1 weiteren Autor:innen.
Abstract (Originalsprache)
Most feature selection methods assess feature relevance using individual scoring criteria followed by a separate selection step. Although clustering-based approaches may at times capture relationships among features, relevance and redundancy are not always addressed in unison. Consequently, redundant features may receive high relevance scores and get selected, possibly leading to less informative feature subsets. This study proposes Large Language Model - embedding-based Semantic Feature Clustering (LLM-SFC), a two-level feature selection pipeline that combines semantic clustering with label-dependent feature selection. In the first level, each feature, expressed as a brief label-agnostic natural-language description, is transformed into a numerical representation using a pretrained sentence-embedding language model. Subsequently, semantic clustering of these embeddings is performed. In the second level, a label-dependent statistical criterion selects one representative feature from each cluster. This reduces redundancy while retaining features relevant to the target label. The proposed pipeline evaluates two clustering methods, Agglomerative Clustering and DBSCAN, and three label-dependent selection statistics. Experiments on three datasets from diverse domains show that agglomerative clustering reduced the feature set by 22.8–55.9% while improving mean accuracy. Statistical significance is observed for 40 of 63 classifier-statistic combinations with no significant losses anywhere. DBSCAN performed comparably when reduction was moderate, but performance dropped under aggressive reduction. Overall, the results demonstrate that semantic clustering can effectively reduce redundancy while preserving or improving predictive performance.
Zitieren
APA
Choudhary, S., & Saxena, B. (2026). Semantic Feature Clustering Using LLM Embeddings for Label-Dependent Feature Selection. International Journal of Advanced Computer Science and Applications, 17(10). https://doi.org/10.14569/ijacsa.2026.0171049
Vancouver
Choudhary S, Saxena B. Semantic Feature Clustering Using LLM Embeddings for Label-Dependent Feature Selection. International Journal of Advanced Computer Science and Applications. 2026;17(10). doi:10.14569/ijacsa.2026.0171049
BibTeX
@article{Choudhary2026,
author = {Choudhary, Shikha and Saxena, Bhawna},
title = {Semantic Feature Clustering Using LLM Embeddings for Label-Dependent Feature Selection},
journal = {International Journal of Advanced Computer Science and Applications},
year = {2026},
volume = {17},
number = {10},
doi = {10.14569/ijacsa.2026.0171049}
}
EUVIMED