Outlier Removal with Weight Penalization and Aggregation: A Robust Variable Selection Method for Enhancing Near-Infrared Spectral Analysis Performance
Analytical Chemistry · 08.04.2025 · 97 (13) · 7325-7332
DOI: 10.1021/acs.analchem.4c07007 ↗Kurz gefasst
Fachartikel in der Zeitschrift Analytical Chemistry, online erschienen am 19.02.2025. Verfasst von Beibei Li und 10 weiteren Autor:innen. Beteiligte Einrichtung: Laboratory of Tobacco Chemistry, Zhengzhou Tobacco Research Institute of CNTC, Zhengzhou 450001, China. Die Arbeit wurde bisher 14-mal zitiert (Crossref) und nennt 40 Literaturangaben.
Abstract (Originalsprache)
Full-wavelength near-infrared (NIR) spectroscopy faces significant challenges due to the strong collinearity among spectral variables and the presence of variables that are highly sensitive to sample fluctuations. Additionally, not all spectral variables contribute equally to the NIR model. Weakly influential variables, although not important on their own, can provide substantial improvement when combined with stronger variables, thus increasing both model stability and prediction accuracy. Therefore, this study proposes a new variable selection method called outlier removal with weight penalization and aggregation (OR-WPA). The method begins by removing outlier spectral variables with high coefficient of variation, which enhances model stability. During the variable selection process, multiple submodels are constructed based on variable subsets, with variable weights assigned according to the absolute values of regression coefficients. A moving window is applied to average the weights, and variables with excessively high weights are penalized, promoting the selection of weakly influential variables that positively contribute to model accuracy. The variable space is iteratively reduced, and the subset of variables associated with the highest predictive accuracy is selected as the final characteristic variable combination. The OR-WPA method was evaluated on three NIR spectral data sets, involving corn, heated tobacco substrate, and flue-cured tobacco. The results were compared with three advanced variable selection methods: Monte Carlo uninformative variable elimination, competitive adaptive reweighted sampling, and bootstrapping soft shrinkage. The results indicate that OR-WPA demonstrates better predictive performance, particularly in predicting low-content components, where it significantly enhances both the accuracy and stability of the NIR model.
Autor:innen und Institutionen
- Laboratory of Tobacco Chemistry, Zhengzhou Tobacco Research Institute of CNTC, Zhengzhou 450001, China.
Zitieren
APA
Li, B., Li, W., Guo, J., Wang, H., Wan, R., Liu, Y., Fan, M., Wang, C., Yang, S., Zhao, L., & Nie, C. (2025). Outlier Removal with Weight Penalization and Aggregation: A Robust Variable Selection Method for Enhancing Near-Infrared Spectral Analysis Performance. Analytical Chemistry, 97(13), 7325-7332. https://doi.org/10.1021/acs.analchem.4c07007
Vancouver
Li B, Li W, Guo J, Wang H, Wan R, Liu Y, et al. Outlier Removal with Weight Penalization and Aggregation: A Robust Variable Selection Method for Enhancing Near-Infrared Spectral Analysis Performance. Analytical Chemistry. 2025;97(13):7325-7332. doi:10.1021/acs.analchem.4c07007
BibTeX
@article{Li2025,
author = {Li, Beibei and Li, Wenting and Guo, Junwei and Wang, Hongbo and Wan, Ran and Liu, Yu and Fan, Meijuan and Wang, Cong and Yang, Song and Zhao, Le and Nie, Cong},
title = {Outlier Removal with Weight Penalization and Aggregation: A Robust Variable Selection Method for Enhancing Near-Infrared Spectral Analysis Performance},
journal = {Analytical Chemistry},
year = {2025},
volume = {97},
number = {13},
pages = {7325--7332},
doi = {10.1021/acs.analchem.4c07007}
}
EUVIMED