BCAT: A Block Causal Transformer for PDE Foundation Models for Fluid Dynamics
Journal of Artificial Intelligence Research · 11.11.2026 · 87
DOI: 10.1613/jair.1.22089 ↗Kurz gefasst
Fachartikel in der Zeitschrift Journal of Artificial Intelligence Research, online erschienen am 11.11.2026. Verfasst von Yuxuan Liu und 2 weiteren Autor:innen.
Abstract (Originalsprache)
Background
Foundation models have demonstrated remarkable generalization in vision and language, but their accurate use for scientific computing remains challenging. For two-dimensional fluid-dynamics prediction, current transformer-based models are often hindered by limited context and rigid causality information, as they typically map fixed windows to the next step.
Objectives
We aim to develop a unified architecture for heterogeneous fluid-dynamics families. The goal is to better capture the complex spatio-temporal dependencies inherent in nonlinear fluid dynamics.
Methods
We introduce BCAT, a block-causal transformer for autoregressive prediction of two-dimensional fluid-dynamics solutions. Our approach adapts block-causal attention to next-frame prediction, leveraging previous frames as contextual priors. The model is trained on 69K trajectory sequences from PDEBench, PDEArena, and CFDBench using the Muon optimizer.
Results
BCAT achieved an average relative 𝐿 2 error of 1.18% across the six evaluated fluid families, reducing error by 45% relative to a 3.4-times-larger baseline. In ablations, next-frame prediction was 3.5x more accurate than the corresponding nexttoken variant. A five-seed analysis, noisy-input stress test, long-rollout analysis, and physics diagnostics further characterize the model.
Conclusions
The block-causal framework captures spatial interactions within frames while enforcing temporal causality across frames. BCAT provides an accurate and computationally efficient model for the two-dimensional fluid-dynamics benchmarks considered here.
Zitieren
APA
Liu, Y., Sun, J., & Schaeffer, H. (2026). BCAT: A Block Causal Transformer for PDE Foundation Models for Fluid Dynamics. Journal of Artificial Intelligence Research, 87. https://doi.org/10.1613/jair.1.22089
Vancouver
Liu Y, Sun J, Schaeffer H. BCAT: A Block Causal Transformer for PDE Foundation Models for Fluid Dynamics. Journal of Artificial Intelligence Research. 2026;87. doi:10.1613/jair.1.22089
BibTeX
@article{Liu2026,
author = {Liu, Yuxuan and Sun, Jingmin and Schaeffer, Hayden},
title = {BCAT: A Block Causal Transformer for PDE Foundation Models for Fluid Dynamics},
journal = {Journal of Artificial Intelligence Research},
year = {2026},
volume = {87},
doi = {10.1613/jair.1.22089}
}
EUVIMED