Video Scene Detection Using Transformer Encoding Linker Network (TELNet)
Sensors · 09.08.2023 · 23 (16) · 7050
DOI: 10.3390/s23167050 ↗Kurz gefasst
Fachartikel in der Zeitschrift Sensors, online erschienen am 09.08.2023. Verfasst von Shu-Ming Tseng und 4 weiteren Autor:innen. Beteiligte Einrichtung: Department of Electronic Engineering, National Taipei University of Technology, Taipei 106335, Taiwan. Die Arbeit wurde bisher 10-mal zitiert (Crossref) und nennt 47 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: the Ministry of Science and Technology, Taiwan, National Science and Technology Council, Taiwan, University of Cincinnati, Cincinnati, OH.
Abstract (Originalsprache)
This paper introduces a transformer encoding linker network (TELNet) for automatically identifying scene boundaries in videos without prior knowledge of their structure. Videos consist of sequences of semantically related shots or chapters, and recognizing scene boundaries is crucial for various video processing tasks, including video summarization. TELNet utilizes a rolling window to scan through video shots, encoding their features extracted from a fine-tuned 3D CNN model (transformer encoder). By establishing links between video shots based on these encoded features (linker), TELNet efficiently identifies scene boundaries where consecutive shots lack links. TELNet was trained on multiple video scene detection datasets and demonstrated results comparable to other state-of-the-art models in standard settings. Notably, in cross-dataset evaluations, TELNet demonstrated significantly improved results (F-score). Furthermore, TELNet’s computational complexity grows linearly with the number of shots, making it highly efficient in processing long videos.
Schlagwörter
Stichwörter der Autor:innen
Autor:innen und Institutionen
- Department of Electronic Engineering, National Taipei University of Technology, Taipei 106335, Taiwan.
- College of Engineering and Applied Science, University of Cincinnati, Cincinnati, OH 45219, USA.
ORCID-Profile: Shu-Ming Tseng, Jia-Bin Chang, Mehdi Norouzi
Förderung
- the Ministry of Science and Technology, Taiwan
- National Science and Technology Council, Taiwan
- University of Cincinnati, Cincinnati, OH
Zitieren
APA
Tseng, S. M., Yeh, Z. T., Wu, C. Y., Chang, J. B., & Norouzi, M. (2023). Video Scene Detection Using Transformer Encoding Linker Network (TELNet). Sensors, 23(16), 7050. https://doi.org/10.3390/s23167050
Vancouver
Tseng SM, Yeh ZT, Wu CY, Chang JB, Norouzi M. Video Scene Detection Using Transformer Encoding Linker Network (TELNet). Sensors. 2023;23(16):7050. doi:10.3390/s23167050
BibTeX
@article{Tseng2023,
author = {Tseng, Shu-Ming and Yeh, Zhi-Ting and Wu, Chia-Yang and Chang, Jia-Bin and Norouzi, Mehdi},
title = {Video Scene Detection Using Transformer Encoding Linker Network (TELNet)},
journal = {Sensors},
year = {2023},
volume = {23},
number = {16},
pages = {7050},
doi = {10.3390/s23167050}
}
EUVIMED