EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

When Agreement Fails: Visual Anchoring and Multimodal Pseudo-Label Reliability in Classroom Behavior Recognition

Yan Ma, Lizhuo Zhang

IEEE Access · 2026 · 14 · 116680-116695

DOI: 10.1109/access.2026.3718349 ↗

Kurz gefasst

Fachartikel in der Zeitschrift IEEE Access, 2026. Verfasst von Yan Ma und 1 weiteren Autor:innen. Die Arbeit nennt 34 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: Education Department of Hunan Province, China.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Dual-model agreement is widely used as a pseudo-label reliability heuristic for multimodal large language models (MLLMs), but whether agreement-filtered labels improve downstream fine-tuning in fine-grained classroom behavior recognition remains uncertain. This study presents a reproducible reliability-audit protocol that evaluates agreement filtering under a single-frame protocol on three SCB classroom behavior sub-datasets (BowTurnHead, HandriseReadWrite, TeacherBehavior), linking bounding-box-level MLLM annotation quality measurement, four filtering strategies, repeated-seed CLIP fine-tuning (linear probe and LoRA), retention-ratio diagnostics, selective-routing analysis, and a ten-model cross-model robustness check spanning three model families (Qwen2, Qwen2.5, Qwen3.5, Qwen3.6, LLaVA, Gemma3, and Gemma4) across 7B–35B parameter scales. Three main findings emerge. First, agreement filtering is not a portable denoising rule: on BowTurnHead, agreement-retained linear probing falls to $15.53\% \pm 1.00\%$ while unfiltered pseudo-labels reach $88.02\% \pm 0.12\%$ , and retention-ratio controls confirm that this collapse reflects class-composition shift and a label-quality ceiling rather than sample count alone. Second, the operative constraint is visual anchoring, the degree to which a category can be recovered from a single frame without discourse context. Categories with low zero-shot CLIP accuracy (e.g., guide, answer) systematically fail across all tested MLLMs; newer-generation models (Qwen3.5-27B, Gemma4-31B) substantially improve on guide (53% and 83% respectively) while answer remains below 19% universally, identifying a task-level single-frame observability boundary. Third, a quality-threshold experiment with Qwen3.5-27B pseudo-labels (annotation accuracy 50.3% vs. Qwen2-VL-7B’s 41.2%) locates the critical annotation accuracy range between 41% and 50% at which pseudo-label fine-tuning crosses the zero-shot baseline (55.29% vs. 53.43%). Together, these findings show that downstream utility is gated by per-class visual anchoring and annotation quality thresholds, which provide a task-internal screening signal for deciding when MLLM pseudo-labels are likely to provide useful supervision in classroom behavior recognition.

Förderung

  • Education Department of Hunan Province, China

Zitieren

APA

Ma, Y., & Zhang, L. (2026). When Agreement Fails: Visual Anchoring and Multimodal Pseudo-Label Reliability in Classroom Behavior Recognition. IEEE Access, 14, 116680-116695. https://doi.org/10.1109/access.2026.3718349

Vancouver

Ma Y, Zhang L. When Agreement Fails: Visual Anchoring and Multimodal Pseudo-Label Reliability in Classroom Behavior Recognition. IEEE Access. 2026;14:116680-116695. doi:10.1109/access.2026.3718349

BibTeX
@article{Ma2026,
  author = {Ma, Yan and Zhang, Lizhuo},
  title = {When Agreement Fails: Visual Anchoring and Multimodal Pseudo-Label Reliability in Classroom Behavior Recognition},
  journal = {IEEE Access},
  year = {2026},
  volume = {14},
  pages = {116680--116695},
  doi = {10.1109/access.2026.3718349}
}
Kostenlos starten →