EUVIMEDEUVIMED entdecken
Crossref

Deep Unsupervised Key Frame Extraction for Efficient Video Classification

Hao Tang, Lei Ding, Songsong Wu, Bin Ren, Nicu Sebe, Paolo Rota

ACM Transactions on Multimedia Computing, Communications, and Applications · 31.08.2023 · 19 (3) · 1-17

DOI: 10.1145/3571735 ↗

Kurz gefasst

Fachartikel in der Zeitschrift ACM Transactions on Multimedia Computing, Communications, and Applications, online erschienen am 25.02.2023. Verfasst von Hao Tang und 5 weiteren Autor:innen. Die Arbeit wurde bisher 49-mal zitiert (Crossref) und nennt 77 Literaturangaben. Gefördert von: PRIN project PREVUE, EU H2020 project AI4Media.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Video processing and analysis have become an urgent task, as a huge amount of videos (e.g., YouTube, Hulu) are uploaded online every day. The extraction of representative key frames from videos is important in video processing and analysis since it greatly reduces computing resources and time. Although great progress has been made recently, large-scale video classification remains an open problem, as the existing methods have not well balanced the performance and efficiency simultaneously. To tackle this problem, this work presents an unsupervised method to retrieve the key frames, which combines the convolutional neural network and temporal segment density peaks clustering. The proposed temporal segment density peaks clustering is a generic and powerful framework, and it has two advantages compared with previous works. One is that it can calculate the number of key frames automatically. The other is that it can preserve the temporal information of the video. Thus, it improves the efficiency of video classification. Furthermore, a long short-term memory network is added on the top of the convolutional neural network to further elevate the performance of classification. Moreover, a weight fusion strategy of different input networks is presented to boost performance. By optimizing both video classification and key frame extraction simultaneously, we achieve better classification performance and higher efficiency. We evaluate our method on two popular datasets (i.e., HMDB51 and UCF101), and the experimental results consistently demonstrate that our strategy achieves competitive performance and efficiency compared with the state-of-the-art approaches.

Förderung

  • PRIN project PREVUE
  • EU H2020 project AI4Media

Zitieren

APA

Tang, H., Ding, L., Wu, S., Ren, B., Sebe, N., & Rota, P. (2023). Deep Unsupervised Key Frame Extraction for Efficient Video Classification. ACM Transactions on Multimedia Computing, Communications, and Applications, 19(3), 1-17. https://doi.org/10.1145/3571735

Vancouver

Tang H, Ding L, Wu S, Ren B, Sebe N, Rota P. Deep Unsupervised Key Frame Extraction for Efficient Video Classification. ACM Transactions on Multimedia Computing, Communications, and Applications. 2023;19(3):1-17. doi:10.1145/3571735

BibTeX
@article{Tang2023,
  author = {Tang, Hao and Ding, Lei and Wu, Songsong and Ren, Bin and Sebe, Nicu and Rota, Paolo},
  title = {Deep Unsupervised Key Frame Extraction for Efficient Video Classification},
  journal = {ACM Transactions on Multimedia Computing, Communications, and Applications},
  year = {2023},
  volume = {19},
  number = {3},
  pages = {1--17},
  doi = {10.1145/3571735}
}
Kostenlos starten →