EUVIMEDEUVIMED entdecken
StudienprotokollOpen Access · CC BY 4.0

TLAssess: A Model-Checking Oracle and Calibration-Gated Scoring Framework for Verified Concurrency-Protocol Assessment

Justinas Janulevičius, Antanas Čenys

Electronics · 14.11.2026 · 15 (22) · 5292

DOI: 10.3390/electronics15225292 ↗

Kurz gefasst

Studienprotokoll in der Zeitschrift Electronics, online erschienen am 14.11.2026. Verfasst von Justinas Janulevičius und 1 weiteren Autor:innen. Die Arbeit nennt 26 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Assessing whether someone understands a concurrency protocol faces two obstacles: ensuring the test’s own answer key is correct and separating real understanding from a lucky guess. We present TLAssess, in which an independent model checker decides every answer’s correctness. Each protocol is modelled as a transition system, and each candidate answer as a temporal-logic proposition whose truth over the reachable state space is decided by the model checker TLC; an item is admitted only when exactly one option, the key, holds. On this verified substrate, we build a scoring layer that reports calibration (Cal) and information (Info) as a pair, with a calibration gate under which a respondent is credited only when both informative and calibrated. We generate verified item banks for mutual exclusion, two-phase commit, and single-decree consensus, and administer them to a five-checkpoint Qwen2.5 ladder. Accuracy rises with model size, but the gap between items read off the specification and inference items widens, while inference accuracy, read from first-token logprobs, shows no detectable departure from chance and falls below it for the smallest model; that confidence signal is uniformly over-sharp and is recovered, in-sample, by a single temperature rescaling. A second model family reproduces the calibration result but not the accuracy result, which proves to depend on the first-token probe, so these findings describe this probe and these families rather than language models in general. All specifications, seeds, banks, and the pinned checker are released.

Zitieren

APA

Janulevičius, J., & Čenys, A. (2026). TLAssess: A Model-Checking Oracle and Calibration-Gated Scoring Framework for Verified Concurrency-Protocol Assessment. Electronics, 15(22), 5292. https://doi.org/10.3390/electronics15225292

Vancouver

Janulevičius J, Čenys A. TLAssess: A Model-Checking Oracle and Calibration-Gated Scoring Framework for Verified Concurrency-Protocol Assessment. Electronics. 2026;15(22):5292. doi:10.3390/electronics15225292

BibTeX
@article{Januleviius2026,
  author = {Janulevičius, Justinas and Čenys, Antanas},
  title = {TLAssess: A Model-Checking Oracle and Calibration-Gated Scoring Framework for Verified Concurrency-Protocol Assessment},
  journal = {Electronics},
  year = {2026},
  volume = {15},
  number = {22},
  pages = {5292},
  doi = {10.3390/electronics15225292}
}
Kostenlos starten →