Arithmetic Feature Interaction Is Necessary for Deep Tabular Learning
Proceedings of the AAAI Conference on Artificial Intelligence · 24.03.2024 · 38 (10) · 11516-11524
DOI: 10.1609/aaai.v38i10.29033 ↗Kurz gefasst
Fachartikel in der Zeitschrift Proceedings of the AAAI Conference on Artificial Intelligence, online erschienen am 24.03.2024. Verfasst von Yi Cheng und 5 weiteren Autor:innen. Die Arbeit wurde bisher 19-mal zitiert (Crossref).
Abstract (Originalsprache)
Until recently, the question of the effective inductive bias of deep models on tabular data has remained unanswered. This paper investigates the hypothesis that arithmetic feature interaction is necessary for deep tabular learning. To test this point, we create a synthetic tabular dataset with a mild feature interaction assumption and examine a modified transformer architecture enabling arithmetical feature interactions, referred to as AMFormer. Results show that AMFormer outperforms strong counterparts in fine-grained tabular data modeling, data efficiency in training, and generalization. This is attributed to its parallel additive and multiplicative attention operators and prompt-based optimization, which facilitate the separation of tabular samples in an extended space with arithmetically-engineered features. Our extensive experiments on real-world data also validate the consistent effectiveness, efficiency, and rationale of AMFormer, suggesting it has established a strong inductive bias for deep learning on tabular data. Code is available at https://github.com/aigc-apps/AMFormer.
Zitieren
APA
Cheng, Y., Hu, R., Ying, H., Shi, X., Wu, J., & Lin, W. (2024). Arithmetic Feature Interaction Is Necessary for Deep Tabular Learning. Proceedings of the AAAI Conference on Artificial Intelligence, 38(10), 11516-11524. https://doi.org/10.1609/aaai.v38i10.29033
Vancouver
Cheng Y, Hu R, Ying H, Shi X, Wu J, Lin W. Arithmetic Feature Interaction Is Necessary for Deep Tabular Learning. Proceedings of the AAAI Conference on Artificial Intelligence. 2024;38(10):11516-11524. doi:10.1609/aaai.v38i10.29033
BibTeX
@article{Cheng2024,
author = {Cheng, Yi and Hu, Renjun and Ying, Haochao and Shi, Xing and Wu, Jian and Lin, Wei},
title = {Arithmetic Feature Interaction Is Necessary for Deep Tabular Learning},
journal = {Proceedings of the AAAI Conference on Artificial Intelligence},
year = {2024},
volume = {38},
number = {10},
pages = {11516--11524},
doi = {10.1609/aaai.v38i10.29033}
}
EUVIMED