EUVIMEDOpdag EUVIMED
Crossref

Second-level Caches: Not for Instructions

Muhammad Hassan, Chang Hyun Park, David Black-Schaffer

ACM Transactions on Architecture and Code Optimization · 2025-12-31 · 22 (4) · 1-18

DOI: 10.1145/3769080 ↗

At a glance

Journal article in ACM Transactions on Architecture and Code Optimization, published online on 2025-12-13. Written by Muhammad Hassan and 2 co-authors. Cited 2 times so far (Crossref); 37 references. Funded by: Knut and Alice Wallenberg Foundation, Wallenberg Academy Fellows Program, Swedish Research Council.

AbstractBibliografiske oplysningerCiteRelated studies

Abstract (original language)

Growing instruction footprints are straining processor front-ends, increasing fetch latency, and causing pipeline stalls. The universal approach to addressing this has been keeping instructions in each level of the cache hierarchy, but a plethora of more advanced techniques, ranging from instruction prefetching to prioritizing instruction cache-lines, have also been proposed to reduce instruction fetch latency. In this work we identify a significant subset of benchmarks that exhibit an insensitivity to instruction fetch latency to the extent that we can take the opposite approach and explicitly bypass the L2 for instructions. While conventional wisdom suggests that this will hurt performance, our detailed analysis of 2,120 traces finds that two factors mitigate this effect. First, the presence of ubiquitous effective instruction prefetching in the form of fetch-directed instruction prefetching (FDIP) means that as long as instructions are present in the LLC, the additional latency of missing in the L2 is often insignificant. And, second, by keeping instructions out of the L2, we find that the increase in data locality in the L2 leads to significant performance gains. Based on our analysis, we propose bypassing the L2 for instructions for a subset of applications and demonstrate how a simple decision tree can be trained offline and used online to cheaply and effectively choose when to bypass. Our technique increases performance by 3.4% geomean for sensitive benchmarks (those that see an improvement of at least 1% with an infinite I-cache) compared to an aggressive FDIP baseline, reaching up to 55% of the perfect I-cache performance with minimal to no overhead.

Funding

  • Knut and Alice Wallenberg Foundation
  • Wallenberg Academy Fellows Program
  • Swedish Research Council
  • Institute of Information & communications Technology Planning & Evaluation (IITP) grant funded by the Korea governmen

Cite

APA

Hassan, M., Park, C. H., & Black-Schaffer, D. (2025). Second-level Caches: Not for Instructions. ACM Transactions on Architecture and Code Optimization, 22(4), 1-18. https://doi.org/10.1145/3769080

Vancouver

Hassan M, Park CH, Black-Schaffer D. Second-level Caches: Not for Instructions. ACM Transactions on Architecture and Code Optimization. 2025;22(4):1-18. doi:10.1145/3769080

BibTeX
@article{Hassan2025,
  author = {Hassan, Muhammad and Park, Chang Hyun and Black-Schaffer, David},
  title = {Second-level Caches: Not for Instructions},
  journal = {ACM Transactions on Architecture and Code Optimization},
  year = {2025},
  volume = {22},
  number = {4},
  pages = {1--18},
  doi = {10.1145/3769080}
}
Start for free →