Perception-Oriented U-Shaped Transformer Network for 360-Degree No-Reference Image Quality Assessment
IEEE Transactions on Broadcasting · 2023-06 · 69 (2) · 396-405
DOI: 10.1109/tbc.2022.3231101 ↗Kurz gefasst
Fachartikel in der Zeitschrift IEEE Transactions on Broadcasting, 2023-06. Verfasst von Mingliang Zhou und 9 weiteren Autor:innen. Die Arbeit wurde bisher 124-mal zitiert (Crossref) und nennt 67 Literaturangaben. Gefördert von: National Natural Science Foundation of China, General Program of the National Natural Science Foundation of Chongqing, Fundamental Research Funds for the Central Universities.
Abstract (Originalsprache)
Generally, 360-degree images have absolute senses of reality and three-dimensionality, providing a wide range of immersive interactions. Due to the novel rendering and display technology of 360-degree images, they have more complex perceptual characteristics than other images. It is challenging to perform comprehensive image quality assessment (IQA) learning by simply stacking multichannel neural network architectures for pre/postprocessing, compression, and rendering tasks. To thoroughly learn the global and local features in 360-degree images, reduce the complexity of multichannel neural network models and simplify the training process, this paper proposes a joint architecture with user perception and an efficient transformer dedicated to 360-degree no-reference (NR) IQA. The input of the proposed method is a 360-degree cube map projection (CMP) image. Furthermore, the proposed 360-degree NRIQA method includes a saliency map-based non-overlapping self-attention selection module and a U-shaped transformer (U-former)-based feature extraction module to account for perceptual region importance and projection distortion. The transformer-based architecture and the weighted average technique are jointly utilized for predicting local perceptual quality. Experimental results obtained on widely used databases show that the proposed model outperforms other state-of-the-art methods in NR 360-degree image quality evaluation cases. Furthermore, a cross-database evaluation and an ablation study also demonstrate the inherent robustness and generalization ability of the proposed model.
Förderung
- National Natural Science Foundation of China
- General Program of the National Natural Science Foundation of Chongqing
- Fundamental Research Funds for the Central Universities
- Human Resources and Social Security Bureau Project of Chongqing
- Huawei Project
- Suzhou Institute of USTC
- Youth Project of Guizhou Education Department
- University-Level Scientific Research Projects
Zitieren
APA
Zhou, M., Chen, L., Wei, X., Liao, X., Mao, Q., Wang, H., Pu, H., Luo, J., Xiang, T., & Fang, B. (2023). Perception-Oriented U-Shaped Transformer Network for 360-Degree No-Reference Image Quality Assessment. IEEE Transactions on Broadcasting, 69(2), 396-405. https://doi.org/10.1109/tbc.2022.3231101
Vancouver
Zhou M, Chen L, Wei X, Liao X, Mao Q, Wang H, et al. Perception-Oriented U-Shaped Transformer Network for 360-Degree No-Reference Image Quality Assessment. IEEE Transactions on Broadcasting. 2023;69(2):396-405. doi:10.1109/tbc.2022.3231101
BibTeX
@article{Zhou2023,
author = {Zhou, Mingliang and Chen, Lei and Wei, Xuekai and Liao, Xingran and Mao, Qin and Wang, Heqiang and Pu, Huayan and Luo, Jun and Xiang, Tao and Fang, Bin},
title = {Perception-Oriented U-Shaped Transformer Network for 360-Degree No-Reference Image Quality Assessment},
journal = {IEEE Transactions on Broadcasting},
year = {2023},
volume = {69},
number = {2},
pages = {396--405},
doi = {10.1109/tbc.2022.3231101}
}
EUVIMED