Do Language Models Prefer Better Demonstrations or Easier Output Tokens?
Computers and Artificial Intelligence · 29.10.2026 · 3 (5) · 1-10
DOI: 10.70267/cai.v3n5.0110 ↗Kurz gefasst
Fachartikel in der Zeitschrift Computers and Artificial Intelligence, online erschienen am 29.10.2026. Verfasst von Jiaye Zhang. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).
Abstract (Originalsprache)
Model feedback is often used to choose in-context demonstrations, but a high probability for the correct answer token may reflect token compatibility rather than useful example content. We audit this issue with Qwen2.5-0.5B-Instruct on BoolQ using corrected inputs, frozen samples, and query-level inference. Under the original Yes/No labels, the preferred same-label demonstration improves the held-out margin over a frozen random baseline by 0.2526. Late-layer residual interventions, however, show no reliable advantage for a matched donor over an unrelated donor with the same label. Counterbalanced A/B mappings then make the raw preferred-minus-random effect follow the output token. A new frozen extension compares raw correct-token likelihood, a binary logit margin, and content-free calibration. The raw selector is harmful on average and strongly favors B. Binary-margin selection gives the largest average held-out gain but leaves a smaller bias toward A. Content-free calibration reduces the raw token gap but does not meet the preset equivalence rule. These results show that model-based demonstration preference depends on the full prompt, label vocabulary, and scoring rule. It should not be read as a direct measure of semantic example quality.
Zitieren
APA
Zhang, J. (2026). Do Language Models Prefer Better Demonstrations or Easier Output Tokens?. Computers and Artificial Intelligence, 3(5), 1-10. https://doi.org/10.70267/cai.v3n5.0110
Vancouver
Zhang J. Do Language Models Prefer Better Demonstrations or Easier Output Tokens?. Computers and Artificial Intelligence. 2026;3(5):1-10. doi:10.70267/cai.v3n5.0110
BibTeX
@article{Zhang2026,
author = {Zhang, Jiaye},
title = {Do Language Models Prefer Better Demonstrations or Easier Output Tokens?},
journal = {Computers and Artificial Intelligence},
year = {2026},
volume = {3},
number = {5},
pages = {1--10},
doi = {10.70267/cai.v3n5.0110}
}
EUVIMED