EUVIMEDEUVIMED entdecken
Open Access · CC BY-NC 4.0

YT-RAG: A Multimodal Retrieval-Augmented Generation Framework for YouTube Video Understanding

Gangadhari Swapna, Chebrolu Yogesh, G. J. S. Hari Teja

International Research Journal on Advanced Science Hub · 15.05.2026 · 8 (04) · 164-170

DOI: 10.47392/irjash.2026.018 ↗

Kurz gefasst

Fachartikel in der Zeitschrift International Research Journal on Advanced Science Hub, online erschienen am 15.05.2026. Verfasst von Gangadhari Swapna und 2 weiteren Autor:innen. Der Volltext ist frei zugänglich (Open Access, CC BY-NC 4.0).

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

The vast body of video content on platforms such as YouTube represents one of the richest yet most query-inaccessible knowledge repositories in existence. We present YT-RAG, a multimodal Retrieval-Augmented Generation (RAG) system that enables natural-language conversation with any YouTube video by independently indexing its spoken transcript as 384-dimensional text embeddings and its visual frames as 512-dimensional image embeddings using FastEmbed, with no GPU infrastructure required. Three principal contributions define the system: (i) a parallel dual-modality ingestion pipeline running transcript and frame extraction as concurrent asyncio tasks; (ii) an agentic retrieval loop powered by Google Gemini 2.0 Flash’s native tool-call API, making retrieval conditional on model judgment rather than mandatory; and (iii) a semantic user-notes channel providing a personalised third retrieval layer absent from all prior video RAG literature. Empirical evaluation across six YouTube content categories — Comedy, Podcast, Cooking, News, Tutorials, and Coding — shows aggregate dual-modality retrieval achieves Hit@5 approximately 4× higher than text-only and 8× higher than image-only retrieval. A systematic anti-correlation in per-category modality strengths confirms that the two channels retrieve complementary evidence. Deployed as a Chrome extension operating natively alongside the YouTube player, YT-RAG is training-free, containerised, and built entirely on official APIs without agentic frameworks

Zitieren

APA

Swapna, G., Yogesh, C., & Teja, G. J. S. H. (2026). YT-RAG: A Multimodal Retrieval-Augmented Generation Framework for YouTube Video Understanding. International Research Journal on Advanced Science Hub, 8(04), 164-170. https://doi.org/10.47392/irjash.2026.018

Vancouver

Swapna G, Yogesh C, Teja GJSH. YT-RAG: A Multimodal Retrieval-Augmented Generation Framework for YouTube Video Understanding. International Research Journal on Advanced Science Hub. 2026;8(04):164-170. doi:10.47392/irjash.2026.018

BibTeX
@article{Swapna2026,
  author = {Swapna, Gangadhari and Yogesh, Chebrolu and Teja, G. J. S. Hari},
  title = {YT-RAG: A Multimodal Retrieval-Augmented Generation Framework for YouTube Video Understanding},
  journal = {International Research Journal on Advanced Science Hub},
  year = {2026},
  volume = {8},
  number = {04},
  pages = {164--170},
  doi = {10.47392/irjash.2026.018}
}
Kostenlos starten →