EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

Carbon-Taxed Transformers: A Green Compression Pipeline for Overgrown Language Models

Ajmain Inqiad Alam, Palash Roy, Chanchal K. Roy, Banani Roy, Kevin A. Schneider

Proceedings of the ACM on Software Engineering · 30.06.2026 · 3 (FSE) · 1035-1058

DOI: 10.1145/3797075 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Proceedings of the ACM on Software Engineering, online erschienen am 30.06.2026. Verfasst von Ajmain Inqiad Alam und 4 weiteren Autor:innen. Die Arbeit nennt 71 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: NSERC.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

The accelerating adoption of Large Language Models (LLMs) in software engineering (SE) has brought with it a silent crisis: unsustainable computational cost. While these models demonstrate remarkable capabilities in different SE tasks, they are unmanageably large, slow to deploy, memory-intensive, and carbon-heavy. This reality threatens not only the scalability and accessibility of AI-powered SE, but also its long-term environmental sustainability. The research challenge is clear: we must go beyond accuracy and address efficiency and environmental cost as first-class design constraints. To meet this challenge, we introduce Carbon-Taxed Transformers (CTT), a systematic multi-architectural compression principled pipeline ordering inspired by economic carbon taxation principles. Drawing from the economic concept of carbon pricing, CTT operationalizes a computational carbon tax that penalizes architectural inefficiencies and rewards deployment-ready compression. We evaluate CTT across three core SE tasks: code clone detection, code summarization, and code generation, with models spanning encoder-only, encoder-decoder, and decoder-only architecture. Our results show that CTT delivers on inference: (1) up to 49× memory reduction, (2) time reduction up to 8-10× for clone detection, up to 3× for summarization, and 4–7× for generation, (3) up to 81% reduction in CO 2 emissions and (4) CTT retains around 98% accuracy on clone detection, around 89% on summarization, and up to 91% (textual metrics) and 68% (pass@1) for generation. Two ablation studies show that pipeline ordering and individual component contributions are both essential, providing empirical justification for CTT’s design and effectiveness. This work establishes a viable path toward responsible AI in SE through aggressive yet performance-preserving compression.

Förderung

  • NSERC

Zitieren

APA

Alam, A. I., Roy, P., Roy, C. K., Roy, B., & Schneider, K. A. (2026). Carbon-Taxed Transformers: A Green Compression Pipeline for Overgrown Language Models. Proceedings of the ACM on Software Engineering, 3(FSE), 1035-1058. https://doi.org/10.1145/3797075

Vancouver

Alam AI, Roy P, Roy CK, Roy B, Schneider KA. Carbon-Taxed Transformers: A Green Compression Pipeline for Overgrown Language Models. Proceedings of the ACM on Software Engineering. 2026;3(FSE):1035-1058. doi:10.1145/3797075

BibTeX
@article{Alam2026,
  author = {Alam, Ajmain Inqiad and Roy, Palash and Roy, Chanchal K. and Roy, Banani and Schneider, Kevin A.},
  title = {Carbon-Taxed Transformers: A Green Compression Pipeline for Overgrown Language Models},
  journal = {Proceedings of the ACM on Software Engineering},
  year = {2026},
  volume = {3},
  number = {FSE},
  pages = {1035--1058},
  doi = {10.1145/3797075}
}
Kostenlos starten →