Decomposing Video Activity Labeling into Human-Inspectable Steps: A Gray-Box Workflow via Narration-Derived Intermediate Outputs
Applied Sciences · 17.11.2026 · 16 (22) · 11363
DOI: 10.3390/app162211363 ↗Kurz gefasst
Fachartikel in der Zeitschrift Applied Sciences, online erschienen am 17.11.2026. Verfasst von Hongryul Ahn und 2 weiteren Autor:innen. Die Arbeit nennt 43 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: Ministry of Education of the Republic of Korea, National Research Foundation of Korea.
Abstract (Originalsprache)
Video activity labeling divides a video into activity segments and assigns a category to each. Manual annotation requires sequential review, while end-to-end AI models may require retraining when the coding task or taxonomy changes. We propose Narration-Mediated Activity Annotation (NMAA), a gray-box workflow that organizes transcript sentences into activity units and compares the resulting annotations with a human reference to evaluate where one activity ends and another begins (B), how activities are classified (T), and which transcript words support each classification (E). Researchers revise the coding protocol based on these results and observed disagreements, reevaluate it, and retain a revision only when its stated purpose is achieved without a confirmed loss in any of the three measures. We evaluated NMAA on 35 Roblox Restaurant Tycoon videos (13.8 h, 16,707 sentences) using a 1200-sentence human reference. The largest stepwise change came from assigning categories sentence by sentence and merging adjacent same-category sentences afterwards, which produced the largest increase in evidence agreement and brought mean activity-unit length close to the human reference. On the held-out test set, the final protocol improved B, T, and E by 0.100, 0.163, and 0.202, respectively, relative to the baseline. The same revision reduced activity-unit length for all three large language models from different providers. When the full corpus was analyzed with the final protocol, Facility and Interior Management accounted for a larger share of the creators’ narrated management time than of textbook topic coverage, whereas Marketing and Promotion accounted for a smaller share.
Förderung
- Ministry of Education of the Republic of Korea
- National Research Foundation of Korea
Zitieren
APA
Ahn, H., Kim, S., & Jo, M. (2026). Decomposing Video Activity Labeling into Human-Inspectable Steps: A Gray-Box Workflow via Narration-Derived Intermediate Outputs. Applied Sciences, 16(22), 11363. https://doi.org/10.3390/app162211363
Vancouver
Ahn H, Kim S, Jo M. Decomposing Video Activity Labeling into Human-Inspectable Steps: A Gray-Box Workflow via Narration-Derived Intermediate Outputs. Applied Sciences. 2026;16(22):11363. doi:10.3390/app162211363
BibTeX
@article{Ahn2026,
author = {Ahn, Hongryul and Kim, Seunghyun and Jo, Mina},
title = {Decomposing Video Activity Labeling into Human-Inspectable Steps: A Gray-Box Workflow via Narration-Derived Intermediate Outputs},
journal = {Applied Sciences},
year = {2026},
volume = {16},
number = {22},
pages = {11363},
doi = {10.3390/app162211363}
}
EUVIMED