Multimodal DeepResearcher: Generating Text-Chart Interleaved Reports from Scratch with Agentic Framework
Proceedings of the AAAI Conference on Artificial Intelligence · 14.03.2026 · 40 (40) · 34368-34377
DOI: 10.1609/aaai.v40i40.40734 ↗Kurz gefasst
Fachartikel in der Zeitschrift Proceedings of the AAAI Conference on Artificial Intelligence, online erschienen am 14.03.2026. Verfasst von Zhaorui Yang und 10 weiteren Autor:innen. Die Arbeit wurde bisher 5-mal zitiert (Crossref).
Abstract (Originalsprache)
Visualizations play a crucial part in effective communication of concepts and information. Recent advances in reasoning and retrieval augmented generation have enabled Large Language Models (LLMs) to perform deep research and generate comprehensive reports. Despite its progress, existing deep research frameworks primarily focus on generating text-only content, leaving the automated generation of interleaved texts and visualizations underexplored. This novel task poses key challenges in designing informative visualizations and effectively integrating them with text reports. To address these challenges, we propose Formal Description of Visualization (FDV), a structured textual representation of charts that enables LLMs to learn from and generate diverse, high-quality visualizations. Building on this representation, we introduce Multimodal DeepResearcher, an agentic framework that decomposes the task into four stages: (1) researching, (2) exemplar report textualization, (3) planning and (4) multimodal report generation. For the evaluation of the generated reports, we develop MultimodalReportBench which contains 100 diverse topics as inputs, and a set of dedicated metrics for report and chart evaluation. Extensive experiments across models and evaluation methods demonstrate the effectiveness of Multimodal DeepResearcher. Notably, utilizing the same Claude 3.7 Sonnet model, Multimodal DeepResearcher achieves an 82% overall win rate over the baseline method.
Zitieren
APA
Yang, Z., Pan, B., Wang, H., Wang, Y., Liu, X., Weng, L., Feng, Y., Feng, H., Zhu, M., Zhang, B., & Chen, W. (2026). Multimodal DeepResearcher: Generating Text-Chart Interleaved Reports from Scratch with Agentic Framework. Proceedings of the AAAI Conference on Artificial Intelligence, 40(40), 34368-34377. https://doi.org/10.1609/aaai.v40i40.40734
Vancouver
Yang Z, Pan B, Wang H, Wang Y, Liu X, Weng L, et al. Multimodal DeepResearcher: Generating Text-Chart Interleaved Reports from Scratch with Agentic Framework. Proceedings of the AAAI Conference on Artificial Intelligence. 2026;40(40):34368-34377. doi:10.1609/aaai.v40i40.40734
BibTeX
@article{Yang2026,
author = {Yang, Zhaorui and Pan, Bo and Wang, Han and Wang, Yiyao and Liu, Xingyu and Weng, Luoxuan and Feng, Yingchaojie and Feng, Haozhe and Zhu, Minfeng and Zhang, Bo and Chen, Wei},
title = {Multimodal DeepResearcher: Generating Text-Chart Interleaved Reports from Scratch with Agentic Framework},
journal = {Proceedings of the AAAI Conference on Artificial Intelligence},
year = {2026},
volume = {40},
number = {40},
pages = {34368--34377},
doi = {10.1609/aaai.v40i40.40734}
}
EUVIMED