Fine-Grained Visual Categorization: A Spatial–Frequency Feature Fusion Perspective
IEEE Transactions on Circuits and Systems for Video Technology · 2023-06 · 33 (6) · 2798-2812
DOI: 10.1109/tcsvt.2022.3227737 ↗Kurz gefasst
Fachartikel in der Zeitschrift IEEE Transactions on Circuits and Systems for Video Technology, 2023-06. Verfasst von Min Wang und 4 weiteren Autor:innen. Die Arbeit wurde bisher 38-mal zitiert (Crossref) und nennt 101 Literaturangaben. Gefördert von: National Natural Science Foundation of China, Natural Science Foundation of Sichuan Province, Open Fund of State Key Laboratory of Oil and Gas Reservoir Geology and Exploitation.
Abstract (Originalsprache)
Fine-grained visual categorization is a challenging issue owing to high intra-class and low inter-class variances. Classical approaches rely on pre-trained models or many fine annotations. In this paper, we observe that spatial and frequency information provides distinct image views, and propose a new spatial–frequency feature fusion (SFFF) perspective to handle this challenging issue. Specifically, we design a heterogeneous feature extraction loss function, construct a global and local fusion SFFF network, and propose an importance-sparsity selection strategy. For feature extraction, we focus on the frequency domain feature learning network, extract fine-grained features, and achieve feature complementarity. For feature selection, we propose importance ranking and sparse regularity to constrain spatial–frequency features. For feature fusion, we design a spatial–frequency loss and an inter-layer switching strategy to achieve local-global collaboration. Comparative experiments were performed on popular fine-grained datasets and classic datasets such as CUB200-2011, Stanford Cars, Stanford Dogs, FGVC-Aircraft, and CIFAR100. The effectiveness and outstanding performance of SFFF are confirmed by comparisons with more than 40 state-of-the-art fine-grained categorization methods. Ablation studies and visualizations are provided to facilitate an understanding of our approach.
Förderung
- National Natural Science Foundation of China
- Natural Science Foundation of Sichuan Province
- Open Fund of State Key Laboratory of Oil and Gas Reservoir Geology and Exploitation
- Science and Technology Cooperation Project of the China National Petroleum Corporation-Southwest Petroleum University (CNPC-SWPU) Innovation Alliance
- Special Project for the Central Government to Guide the Development of Local Science and Technology in Sichuan Province
Zitieren
APA
Wang, M., Zhao, P., Lu, X., Min, F., & Wang, X. (2023). Fine-Grained Visual Categorization: A Spatial–Frequency Feature Fusion Perspective. IEEE Transactions on Circuits and Systems for Video Technology, 33(6), 2798-2812. https://doi.org/10.1109/tcsvt.2022.3227737
Vancouver
Wang M, Zhao P, Lu X, Min F, Wang X. Fine-Grained Visual Categorization: A Spatial–Frequency Feature Fusion Perspective. IEEE Transactions on Circuits and Systems for Video Technology. 2023;33(6):2798-2812. doi:10.1109/tcsvt.2022.3227737
BibTeX
@article{Wang2023,
author = {Wang, Min and Zhao, Peng and Lu, Xin and Min, Fan and Wang, Xizhao},
title = {Fine-Grained Visual Categorization: A Spatial–Frequency Feature Fusion Perspective},
journal = {IEEE Transactions on Circuits and Systems for Video Technology},
year = {2023},
volume = {33},
number = {6},
pages = {2798--2812},
doi = {10.1109/tcsvt.2022.3227737}
}
EUVIMED