Equi-Tuning: Group Equivariant Fine-Tuning of Pretrained Models
Proceedings of the AAAI Conference on Artificial Intelligence · 26.06.2023 · 37 (6) · 6788-6796
DOI: 10.1609/aaai.v37i6.25832 ↗Kurz gefasst
Fachartikel in der Zeitschrift Proceedings of the AAAI Conference on Artificial Intelligence, online erschienen am 26.06.2023. Verfasst von Sourya Basu und 6 weiteren Autor:innen. Die Arbeit wurde bisher 6-mal zitiert (Crossref).
Abstract (Originalsprache)
We introduce equi-tuning, a novel fine-tuning method that transforms (potentially non-equivariant) pretrained models into group equivariant models while incurring minimum L_2 loss between the feature representations of the pretrained and the equivariant models. Large pretrained models can be equi-tuned for different groups to satisfy the needs of various downstream tasks. Equi-tuned models benefit from both group equivariance as an inductive bias and semantic priors from pretrained models. We provide applications of equi-tuning on three different tasks: image classification, compositional generalization in language, and fairness in natural language generation (NLG). We also provide a novel group-theoretic definition for fairness in NLG. The effectiveness of this definition is shown by testing it against a standard empirical method of fairness in NLG. We provide experimental results for equi-tuning using a variety of pretrained models: Alexnet, Resnet, VGG, and Densenet for image classification; RNNs, GRUs, and LSTMs for compositional generalization; and GPT2 for fairness in NLG. We test these models on benchmark datasets across all considered tasks to show the generality and effectiveness of the proposed method.
Zitieren
APA
Basu, S., Sattigeri, P., Ramamurthy, K. N., Chenthamarakshan, V., Varshney, K. R., Varshney, L. R., & Das, P. (2023). Equi-Tuning: Group Equivariant Fine-Tuning of Pretrained Models. Proceedings of the AAAI Conference on Artificial Intelligence, 37(6), 6788-6796. https://doi.org/10.1609/aaai.v37i6.25832
Vancouver
Basu S, Sattigeri P, Ramamurthy KN, Chenthamarakshan V, Varshney KR, Varshney LR, et al. Equi-Tuning: Group Equivariant Fine-Tuning of Pretrained Models. Proceedings of the AAAI Conference on Artificial Intelligence. 2023;37(6):6788-6796. doi:10.1609/aaai.v37i6.25832
BibTeX
@article{Basu2023,
author = {Basu, Sourya and Sattigeri, Prasanna and Ramamurthy, Karthikeyan Natesan and Chenthamarakshan, Vijil and Varshney, Kush R. and Varshney, Lav R. and Das, Payel},
title = {Equi-Tuning: Group Equivariant Fine-Tuning of Pretrained Models},
journal = {Proceedings of the AAAI Conference on Artificial Intelligence},
year = {2023},
volume = {37},
number = {6},
pages = {6788--6796},
doi = {10.1609/aaai.v37i6.25832}
}
EUVIMED