EUVIMEDEUVIMED entdecken
Crossref

SoMe: A Realistic Benchmark for LLM-based Social Media Agents

Dizhan Xue, Jing Cui, Shengsheng Qian, Chuanrui Hu, Changsheng Xu

Proceedings of the AAAI Conference on Artificial Intelligence · 14.03.2026 · 40 (2) · 1391-1399

DOI: 10.1609/aaai.v40i2.37113 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Proceedings of the AAAI Conference on Artificial Intelligence, online erschienen am 14.03.2026. Verfasst von Dizhan Xue und 4 weiteren Autor:innen.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Intelligent agents powered by large language models (LLMs) have recently demonstrated impressive capabilities and gained increasing popularity on social media platforms. While LLM agents are reshaping the ecology of social media, there exists a current gap in conducting a comprehensive evaluation of their ability to comprehend media content, understand user behaviors, and make intricate decisions. To address this challenge, we introduce SoMe, a pioneering benchmark designed to evaluate social media agents equipped with various agent tools for accessing and analyzing social media data. SoMe comprises a diverse collection of 8 social media agent tasks, 9,164,284 posts, 6,591 user profiles, and 25,686 reports from various social media platforms and external websites, with 17,869 meticulously annotated task queries. Compared with the existing datasets and benchmarks for social media tasks, SoMe is the first to provide a versatile and realistic platform for LLM-based social media agents to handle diverse social media tasks. By extensive quantitative and qualitative analysis, we provide the first overview insight into the performance of mainstream agentic LLMs in realistic social media environments and identify several limitations. Our evaluation reveals that both the current closed-source and open-source LLMs cannot handle social media agent tasks satisfactorily. SoMe provides a challenging yet meaningful testbed for future social media agents.

Zitieren

APA

Xue, D., Cui, J., Qian, S., Hu, C., & Xu, C. (2026). SoMe: A Realistic Benchmark for LLM-based Social Media Agents. Proceedings of the AAAI Conference on Artificial Intelligence, 40(2), 1391-1399. https://doi.org/10.1609/aaai.v40i2.37113

Vancouver

Xue D, Cui J, Qian S, Hu C, Xu C. SoMe: A Realistic Benchmark for LLM-based Social Media Agents. Proceedings of the AAAI Conference on Artificial Intelligence. 2026;40(2):1391-1399. doi:10.1609/aaai.v40i2.37113

BibTeX
@article{Xue2026,
  author = {Xue, Dizhan and Cui, Jing and Qian, Shengsheng and Hu, Chuanrui and Xu, Changsheng},
  title = {SoMe: A Realistic Benchmark for LLM-based Social Media Agents},
  journal = {Proceedings of the AAAI Conference on Artificial Intelligence},
  year = {2026},
  volume = {40},
  number = {2},
  pages = {1391--1399},
  doi = {10.1609/aaai.v40i2.37113}
}
Kostenlos starten →