Can Multi-Agent LLMs Identify Their Peers? Stylometric Fingerprinting in Role-Constrained Political Analysis
Trends in Computer Science and Information Technology · 30.05.2026 · 11 (1) · 58-67
DOI: 10.17352/tcsit.000111 ↗Kurz gefasst
Fachartikel in der Zeitschrift Trends in Computer Science and Information Technology, online erschienen am 30.05.2026. Verfasst von Juergen Dietrich. Die Arbeit nennt 15 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).
Abstract (Originalsprache)
Multi-agent large language model (LLM) pipelines for political statement analysis are vulnerable to peer-preservation bias: models tend to protect peer models from deactivation and show identity-dependent scoring distortions. In plain terms, one model may recognise a peer's identity even when explicit identifiers are hidden, and may adjust its scoring accordingly. Prompt-level anonymization was proposed as a mitigation, but prior work simultaneously documented that stylometric fingerprints survive anonymization in role-constrained outputs, raising the question of whether this mitigation is sufficient. This paper provides the first systematic investigation of whether LLMs can identify the model family behind political analysis texts under anonymization conditions. We evaluate three classifier approaches: LLM zero-shot and few-shot (Claude Sonnet 4.6 and Llama-3.3-70B) and a ne-tuned T5-base model on a ve-class attribution task covering four commercial LLM families and an open-world `unknown' class. We introduce a statement-disjoint cross-validation protocol (SD-CV; de ned in Section 3.5) that guarantees no content overlap between training and validation data, and contrast it with a run-disjoint baseline (RD-CV). T5 achieves Macro F1 = 0.991 (±0.008) under SD-CV and F1 = 0.978 on 24 completely held-out statements, robust despite a 2.1× increase in train-test content distance versus RD-CV (0.767 vs. 0.366, p < 0.001), demonstrating genuine stylometric generalization. A fractional SD-CV analysis identifies a performance knee at 40% of training data (≈440 texts). Our findings confirm that prompt-level anonymization alone cannot neutralize model identity signals, with direct implications for EU AI Act compliance (Articles 13, 14, 26) and for computer system validation (CSV) in quality-critical multi-agent deployments.
Zitieren
APA
Dietrich, J. (2026). Can Multi-Agent LLMs Identify Their Peers? Stylometric Fingerprinting in Role-Constrained Political Analysis. Trends in Computer Science and Information Technology, 11(1), 58-67. https://doi.org/10.17352/tcsit.000111
Vancouver
Dietrich J. Can Multi-Agent LLMs Identify Their Peers? Stylometric Fingerprinting in Role-Constrained Political Analysis. Trends in Computer Science and Information Technology. 2026;11(1):58-67. doi:10.17352/tcsit.000111
BibTeX
@article{Dietrich2026,
author = {Dietrich, Juergen},
title = {Can Multi-Agent LLMs Identify Their Peers? Stylometric Fingerprinting in Role-Constrained Political Analysis},
journal = {Trends in Computer Science and Information Technology},
year = {2026},
volume = {11},
number = {1},
pages = {58--67},
doi = {10.17352/tcsit.000111}
}
EUVIMED