EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

Do Language Models Prefer Better Demonstrations or Easier Output Tokens?

Jiaye Zhang

Computers and Artificial Intelligence · 29.10.2026 · 3 (5) · 1-10

DOI: 10.70267/cai.v3n5.0110 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Computers and Artificial Intelligence, online erschienen am 29.10.2026. Verfasst von Jiaye Zhang. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Model feedback is often used to choose in-context demonstrations, but a high probability for the correct answer token may reflect token compatibility rather than useful example content. We audit this issue with Qwen2.5-0.5B-Instruct on BoolQ using corrected inputs, frozen samples, and query-level inference. Under the original Yes/No labels, the preferred same-label demonstration improves the held-out margin over a frozen random baseline by 0.2526. Late-layer residual interventions, however, show no reliable advantage for a matched donor over an unrelated donor with the same label. Counterbalanced A/B mappings then make the raw preferred-minus-random effect follow the output token. A new frozen extension compares raw correct-token likelihood, a binary logit margin, and content-free calibration. The raw selector is harmful on average and strongly favors B. Binary-margin selection gives the largest average held-out gain but leaves a smaller bias toward A. Content-free calibration reduces the raw token gap but does not meet the preset equivalence rule. These results show that model-based demonstration preference depends on the full prompt, label vocabulary, and scoring rule. It should not be read as a direct measure of semantic example quality.

Zitieren

APA

Zhang, J. (2026). Do Language Models Prefer Better Demonstrations or Easier Output Tokens?. Computers and Artificial Intelligence, 3(5), 1-10. https://doi.org/10.70267/cai.v3n5.0110

Vancouver

Zhang J. Do Language Models Prefer Better Demonstrations or Easier Output Tokens?. Computers and Artificial Intelligence. 2026;3(5):1-10. doi:10.70267/cai.v3n5.0110

BibTeX
@article{Zhang2026,
  author = {Zhang, Jiaye},
  title = {Do Language Models Prefer Better Demonstrations or Easier Output Tokens?},
  journal = {Computers and Artificial Intelligence},
  year = {2026},
  volume = {3},
  number = {5},
  pages = {1--10},
  doi = {10.70267/cai.v3n5.0110}
}
Kostenlos starten →