Prompt Disentanglement via Language Guidance and Representation Alignment for Domain Generalization
IEEE Transactions on Pattern Analysis and Machine Intelligence · 2026-06 · 48 (6) · 6799-6816
DOI: 10.1109/tpami.2026.3661049 ↗Kurz gefasst
Fachartikel in der Zeitschrift IEEE Transactions on Pattern Analysis and Machine Intelligence, 2026-06. Verfasst von De Cheng und 5 weiteren Autor:innen. Die Arbeit wurde bisher 4-mal zitiert (Crossref) und nennt 80 Literaturangaben. Gefördert von: New Generation Artificial Intelligence-National Science and Technology Major Project, National Natural Science Foundation of China, Key R&D Program of Shaanxi Province.
Abstract (Originalsprache)
Domain Generalization (DG) seeks to develop models that perform well on unseen target domains by learning domain-invariant representations. Recent advances in pre-trained Visual Foundation Models (VFMs), such as CLIP, have shown strong potential for enhancing DG through prompt tuning. However, existing VFM-based prompt tuning methods often focus on task-specific adaptation rather than disentangling domain-invariant features, leaving cross-domain generalization insufficiently explored. In this paper, we address this challenge by fully leveraging the controllable and flexible language prompt in VFMs. Observing that the text modality is inherently rich in semantics and easier to disentangle, we propose a novel framework termed Prompt Disentanglement via Language Guidance and Representation Alignment (PADG). PADG first employs a large language model (LLM) to disentangle textual prompts into domain-invariant and domain-specific components, which then guide the learning of domain-invariant visual representations. To complement the limitations of text-only guidance, we further introduce the Worst Explicit Representation Alignment (WERA) module, which enhances visual invariance by simulating bounded domain shifts through learnable stylization prompts and aligning representations between original and perturbed samples. Extensive experiments on mainstream DG benchmarks, including PACS, VLCS, OfficeHome, DomainNet, and TerraInc, demonstrate that PADG consistently outperforms existing state-of-the-art methods, validating its effectiveness in robust domain-invariant representation learning.
Förderung
- New Generation Artificial Intelligence-National Science and Technology Major Project
- National Natural Science Foundation of China
- Key R&D Program of Shaanxi Province
- Scientific and Technological Innovation Teams in Shaanxi Province
- Shaanxi Province Core Technology Research and Development Project
- Fundamental Research Funds for the Central Universities
- CCF-Kuaishou Large Model Explorer Fund
Zitieren
APA
Cheng, Xu, Z., Jiang, X., Li, D., Wang, N., & Gao, X. (2026). Prompt Disentanglement via Language Guidance and Representation Alignment for Domain Generalization. IEEE Transactions on Pattern Analysis and Machine Intelligence, 48(6), 6799-6816. https://doi.org/10.1109/tpami.2026.3661049
Vancouver
Cheng, Xu Z, Jiang X, Li D, Wang N, Gao X. Prompt Disentanglement via Language Guidance and Representation Alignment for Domain Generalization. IEEE Transactions on Pattern Analysis and Machine Intelligence. 2026;48(6):6799-6816. doi:10.1109/tpami.2026.3661049
BibTeX
@article{Cheng2026,
author = {Cheng, De and Xu, Zhipeng and Jiang, Xinyang and Li, Dongsheng and Wang, Nannan and Gao, Xinbo},
title = {Prompt Disentanglement via Language Guidance and Representation Alignment for Domain Generalization},
journal = {IEEE Transactions on Pattern Analysis and Machine Intelligence},
year = {2026},
volume = {48},
number = {6},
pages = {6799--6816},
doi = {10.1109/tpami.2026.3661049}
}
EUVIMED