A Multimodal Agent Framework for Construction Scenarios: Accurate Perception, Dynamic Retrieval, and Explainable Hazard Reasoning
Buildings · 09.12.2025 · 15 (24) · 4439
DOI: 10.3390/buildings15244439 ↗Kurz gefasst
Fachartikel in der Zeitschrift Buildings, online erschienen am 09.12.2025. Verfasst von Sihan Cheng und 3 weiteren Autor:innen. Die Arbeit wurde bisher 6-mal zitiert (Crossref) und nennt 39 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).
Abstract (Originalsprache)
Construction sites are complex environments where traditional safety monitoring methods often suffer from low detection accuracy and limited interpretability. To address these challenges, this study proposes a modular multimodal agent framework that integrates computer vision, knowledge representation, and large language model (LLM)–based reasoning. First, the CLIP model fine-tuned with Low-Rank Adaptation (LoRA) is combined with YOLOv10 to achieve precise recognition of construction activities and personal protective equipment (PPE). Second, a construction safety knowledge graph integrating Retrieval-Augmented Generation (RAG) is constructed to provide structured domain knowledge and enhance contextual understanding. Third, the FusedChain prompting strategy is designed to guide large language models (LLMs) to perform step-by-step safety risk reasoning. Experimental results show that the proposed approach achieves 97.35% accuracy in activity recognition, an average F1-score of 0.84 in PPE detection, and significantly higher performance than existing methods in hazard reasoning. The modular design also facilitates scalable integration with more advanced foundation models, indicating strong potential for real-world deployment in intelligent construction safety management.
Zitieren
APA
Cheng, S., Qi, Y., Wu, R., & Guan, Y. (2025). A Multimodal Agent Framework for Construction Scenarios: Accurate Perception, Dynamic Retrieval, and Explainable Hazard Reasoning. Buildings, 15(24), 4439. https://doi.org/10.3390/buildings15244439
Vancouver
Cheng S, Qi Y, Wu R, Guan Y. A Multimodal Agent Framework for Construction Scenarios: Accurate Perception, Dynamic Retrieval, and Explainable Hazard Reasoning. Buildings. 2025;15(24):4439. doi:10.3390/buildings15244439
BibTeX
@article{Cheng2025,
author = {Cheng, Sihan and Qi, Yujun and Wu, Rui and Guan, Yangyang},
title = {A Multimodal Agent Framework for Construction Scenarios: Accurate Perception, Dynamic Retrieval, and Explainable Hazard Reasoning},
journal = {Buildings},
year = {2025},
volume = {15},
number = {24},
pages = {4439},
doi = {10.3390/buildings15244439}
}
EUVIMED