From Solo Post to Shared Space: How a Public LLM Agent Reshapes Human-to-Human Conversation Structure on a Social Platform

Vollständiger Abstract

Worum geht es in dieser Arbeit?

As large language model (LLM) agents become routine participants in public online conversation, understanding how their replies reshape human-to-human interaction structure is a central challenge for social computing and platform governance. Yet most post-deployment evaluations focus on toxicity or engagement volume, leaving thread-level conversational structure—a dimension critical to deliberative quality, social capital formation, and equality of voice—largely unmeasured. We address this gap through a large-scale structural evaluation of @CommentR , a production LLM agent serving millions of users on Weibo, China’s leading microblogging platform. Modeling each thread as a directed reply graph, we match over 216,000 real post-deployment threads on strictly pre-anchor covariates and estimate effects on human-only conversational structure using a doubly robust estimator. Because agent replies often arrive before any human-to-human interaction is observed, we introduce lifecycle-aware estimands that distinguish early-stage formation effects from mature-thread rewiring effects. Under conditional ignorability, agent replies reduce reciprocity, increase branching, and reduce geographic homophily in early-stage threads, while degree-corrected bridging remains unchanged—consistent with a hub-and-spoke shift from dialogic exchange toward one-off commenting around a focal reply. In mature threads, eligibility for the incumbent-rewiring analysis is itself reduced by the agent—sustained incumbent exchange becomes less likely—so we report the mature-thread rewiring estimates as bounded rather than point-identified and treat the formation regime as the one in which our structural evidence is secure. The magnitude of the reshaping depends on how the agent answers: more comprehensive, factual replies produce a larger focal shift. Reply-target analysis confirms that a substantial share of human comments redirect toward the agent, reducing lateral human-to-human exchange. These findings demonstrate that public AI agents can reshape not only what people say but how people talk to each other , with direct implications for platform governance, conversational agent design, and the structural monitoring of AI-mediated public discourse.