EUVIMEDEUVIMED entdecken
Crossref

VNVC: A Versatile Neural Video Coding Framework for Efficient Human-Machine Vision

Xihua Sheng, Li Li, Dong Liu, Houqiang Li

IEEE Transactions on Pattern Analysis and Machine Intelligence · 2024-07 · 46 (7) · 4579-4596

DOI: 10.1109/tpami.2024.3356548 ↗

Kurz gefasst

Fachartikel in der Zeitschrift IEEE Transactions on Pattern Analysis and Machine Intelligence, 2024-07. Verfasst von Xihua Sheng und 3 weiteren Autor:innen. Hauptthemen laut MEDLINE: Video Recording, Algorithms, Image Processing, Computer-Assisted, Data Compression. Die Arbeit wurde bisher 38-mal zitiert (Crossref) und nennt 78 Literaturangaben. Gefördert von: National Natural Science Foundation of China, Fundamental Research Funds for the Central Universities, GPU cluster built.

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Almost all digital videos are coded into compact representations before being transmitted. Such compact representations need to be decoded back to pixels before being displayed to humans and - as usual - before being enhanced/analyzed by machine vision algorithms. Intuitively, it is more efficient to enhance/analyze the coded representations directly without decoding them into pixels. Therefore, we propose a versatile neural video coding (VNVC) framework, which targets learning compact representations to support both reconstruction and direct enhancement/analysis, thereby being versatile for both human and machine vision. Our VNVC framework has a feature-based compression loop. In the loop, one frame is encoded into compact representations and decoded to an intermediate feature that is obtained before performing reconstruction. The intermediate feature can be used as reference in motion compensation and motion estimation through feature-based temporal context mining and cross-domain motion encoder-decoder to compress the following frames. The intermediate feature is directly fed into video reconstruction, video enhancement, and video analysis networks to evaluate its effectiveness. The evaluation shows that our framework with the intermediate feature achieves high compression efficiency for video reconstruction and satisfactory task performances with lower complexities.

Schlagwörter

MeSH-Schlagwörter (MEDLINE)

Video RecordingAlgorithmsImage Processing, Computer-AssistedData CompressionNeural Networks, ComputerHumans

Förderung

  • National Natural Science Foundation of China
  • Fundamental Research Funds for the Central Universities
  • GPU cluster built
  • MCC Lab of Information Science and Technology Institution
  • University of Science and Technology of China

Zitieren

APA

Sheng, X., Li, L., Liu, D., & Li, H. (2024). VNVC: A Versatile Neural Video Coding Framework for Efficient Human-Machine Vision. IEEE Transactions on Pattern Analysis and Machine Intelligence, 46(7), 4579-4596. https://doi.org/10.1109/tpami.2024.3356548

Vancouver

Sheng X, Li L, Liu D, Li H. VNVC: A Versatile Neural Video Coding Framework for Efficient Human-Machine Vision. IEEE Transactions on Pattern Analysis and Machine Intelligence. 2024;46(7):4579-4596. doi:10.1109/tpami.2024.3356548

BibTeX
@article{Sheng2024,
  author = {Sheng, Xihua and Li, Li and Liu, Dong and Li, Houqiang},
  title = {VNVC: A Versatile Neural Video Coding Framework for Efficient Human-Machine Vision},
  journal = {IEEE Transactions on Pattern Analysis and Machine Intelligence},
  year = {2024},
  volume = {46},
  number = {7},
  pages = {4579--4596},
  doi = {10.1109/tpami.2024.3356548}
}
Kostenlos starten →