Foundation Models Defining a New Era in Vision: A Survey and Outlook
IEEE Transactions on Pattern Analysis and Machine Intelligence · 2025-04 · 47 (4) · 2245-2264
DOI: 10.1109/tpami.2024.3506283 ↗Kurz gefasst
Fachartikel in der Zeitschrift IEEE Transactions on Pattern Analysis and Machine Intelligence, 2025-04. Verfasst von Muhammad Awais und 7 weiteren Autor:innen. Die Arbeit wurde bisher 355-mal zitiert (Crossref) und nennt 221 Literaturangaben.
Abstract (Originalsprache)
Vision systems that see and reason about the compositional nature of visual scenes are fundamental to understanding our world. The complex relations between objects and their locations, ambiguities, and variations in the real-world environment can be better described in human language, naturally governed by grammatical rules and other modalities such as audio and depth. The models learned to bridge the gap between such modalities and large-scale training data facilitate contextual reasoning, generalization, and prompt capabilities at test time. These models are referred to as foundation models. The output of such models can be modified through human-provided prompts without retraining, e.g., segmenting a particular object by providing a bounding box, having interactive dialogues by asking questions about an image or video scene or manipulating the robot's behavior through language instructions. In this survey, we provide a comprehensive review of such emerging foundation models, including typical architecture designs to combine different modalities (vision, text, audio, etc.), training objectives (contrastive, generative), pre-training datasets, fine-tuning mechanisms, and the common prompting patterns; textual, visual, and heterogeneous. We discuss the open challenges and research directions for foundation models in computer vision, including difficulties in their evaluations and benchmarking, gaps in their real-world understanding, limitations of contextual understanding, biases, vulnerability to adversarial attacks, and interpretability issues. We review recent developments in this field, covering a wide range of applications of foundation models systematically and comprehensively.
Zitieren
APA
Awais, M., Naseer, M., Khan, S., Anwer, R. M., Cholakkal, H., Shah, M., Yang, M. H., & Khan, F. S. (2025). Foundation Models Defining a New Era in Vision: A Survey and Outlook. IEEE Transactions on Pattern Analysis and Machine Intelligence, 47(4), 2245-2264. https://doi.org/10.1109/tpami.2024.3506283
Vancouver
Awais M, Naseer M, Khan S, Anwer RM, Cholakkal H, Shah M, et al. Foundation Models Defining a New Era in Vision: A Survey and Outlook. IEEE Transactions on Pattern Analysis and Machine Intelligence. 2025;47(4):2245-2264. doi:10.1109/tpami.2024.3506283
BibTeX
@article{Awais2025,
author = {Awais, Muhammad and Naseer, Muzammal and Khan, Salman and Anwer, Rao Muhammad and Cholakkal, Hisham and Shah, Mubarak and Yang, Ming-Hsuan and Khan, Fahad Shahbaz},
title = {Foundation Models Defining a New Era in Vision: A Survey and Outlook},
journal = {IEEE Transactions on Pattern Analysis and Machine Intelligence},
year = {2025},
volume = {47},
number = {4},
pages = {2245--2264},
doi = {10.1109/tpami.2024.3506283}
}
EUVIMED