EUVIMEDEUVIMED entdecken
Crossref

SmartSight: Mitigating Hallucination in Video-LLMs Without Compromising Video Understanding via Temporal Attention Collapse

Yiming Sun, Mi Zhang, Feifei Li, Geng Hong, Min Yang

Proceedings of the AAAI Conference on Artificial Intelligence · 14.03.2026 · 40 (11) · 9251-9259

DOI: 10.1609/aaai.v40i11.37883 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Proceedings of the AAAI Conference on Artificial Intelligence, online erschienen am 14.03.2026. Verfasst von Yiming Sun und 4 weiteren Autor:innen. Die Arbeit wurde bisher 1-mal zitiert (Crossref).

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Despite Video Large Language Models (Video-LLMs) having rapidly advanced in recent years, perceptual hallucinations pose a substantial safety risk, which severely restricts their real-world applicability. While several methods for hallucination mitigation have been proposed, they often compromise the model’s capacity for video understanding and reasoning. In this work, we propose SmartSight, a pioneering step to address this issue in a training-free manner by leveraging the model’s own introspective capabilities. Specifically, SmartSight generates multiple candidate responses to uncover low-hallucinated outputs that are often obscured by standard greedy decoding. It assesses the hallucination of each response using the Temporal Attention Collapse score, which measures whether the model over-focuses on trivial temporal regions of the input video when generating the response. To improve efficiency, SmartSight identifies the Visual Attention Vanishing point, enabling more accurate hallucination estimation and early termination of hallucinated responses, leading to a substantial reduction in decoding cost. Experiments show that SmartSight substantially lowers hallucinations for QwenVL-2.5-7B by 10.59% on VRIPT-HAL, while simultaneously enhancing video understanding and reasoning, boosting performance on VideoMMMU by 8.86%. These results highlight SmartSight’s effectiveness in improving the reliability of open-source Video-LLMs.

Zitieren

APA

Sun, Y., Zhang, M., Li, F., Hong, G., & Yang, M. (2026). SmartSight: Mitigating Hallucination in Video-LLMs Without Compromising Video Understanding via Temporal Attention Collapse. Proceedings of the AAAI Conference on Artificial Intelligence, 40(11), 9251-9259. https://doi.org/10.1609/aaai.v40i11.37883

Vancouver

Sun Y, Zhang M, Li F, Hong G, Yang M. SmartSight: Mitigating Hallucination in Video-LLMs Without Compromising Video Understanding via Temporal Attention Collapse. Proceedings of the AAAI Conference on Artificial Intelligence. 2026;40(11):9251-9259. doi:10.1609/aaai.v40i11.37883

BibTeX
@article{Sun2026,
  author = {Sun, Yiming and Zhang, Mi and Li, Feifei and Hong, Geng and Yang, Min},
  title = {SmartSight: Mitigating Hallucination in Video-LLMs Without Compromising Video Understanding via Temporal Attention Collapse},
  journal = {Proceedings of the AAAI Conference on Artificial Intelligence},
  year = {2026},
  volume = {40},
  number = {11},
  pages = {9251--9259},
  doi = {10.1609/aaai.v40i11.37883}
}
Kostenlos starten →