EUVIMEDEUVIMED entdecken
Open Access · CC BY-NC 4.0

Bulk‐Switching Memristor‐Based Compute‐In‐Memory Module for Deep Neural Network Training

Yuting Wu, Qiwen Wang, Ziyu Wang, Xinxin Wang, Buvna Ayyagari, Siddarth Krishnan, Michael Chudzik, Wei D Lu iD

Advanced Materials · 2023-11 · 35 (46)

DOI: 10.1002/adma.202305465 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Advanced Materials, online erschienen am 15.10.2023. Verfasst von Yuting Wu und 7 weiteren Autor:innen. Beteiligte Einrichtung: Department of Electrical Engineering and Computer Science, University of Michigan, Ann Arbor, MI, 48109, USA. Die Arbeit wurde bisher 57-mal zitiert (Crossref) und nennt 54 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY-NC 4.0). Gefördert von: National Science Foundation, Applied Materials, Inc..

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

The constant drive to achieve higher performance in deep neural networks (DNNs) has led to the proliferation of very large models. Model training, however, requires intensive computation time and energy. Memristor‐based compute‐in‐memory (CIM) modules can perform vector‐matrix multiplication (VMM) in place and in parallel, and have shown great promises in DNN inference applications. However, CIM‐based model training faces challenges due to non‐linear weight updates, device variations, and low‐precision. In this work, a mixed‐precision training scheme is experimentally implemented to mitigate these effects using a bulk‐switching memristor‐based CIM module. Low‐precision CIM modules are used to accelerate the expensive VMM operations, with high‐precision weight updates accumulated in digital units. Memristor devices are only changed when the accumulated weight update value exceeds a pre‐defined threshold. The proposed scheme is implemented with a system‐onchip of fully integrated analog CIM modules and digital sub‐systems, showing fast convergence of LeNet training to 97.73%. The efficacy of training larger models is evaluated using realistic hardware parameters and verifies that CIM modules can enable efficient mix‐precision DNN training with accuracy comparable to full‐precision software‐trained models. Additionally, models trained on chip are inherently robust to hardware variations, allowing direct mapping to CIM inference chips without additional re‐training.

Schlagwörter

Stichwörter der Autor:innen

deep neural network trainingdeep neural networksin-memory computingmemristorsmixed precision training

Autor:innen und Institutionen

  • Department of Electrical Engineering and Computer Science, University of Michigan, Ann Arbor, MI, 48109, USA.
  • Applied Materials, Inc., Santa Clara, CA, 95054, USA.

ORCID-Profile: Wei D Lu

Förderung

  • National Science Foundation
  • Applied Materials, Inc.

Zitieren

APA

Wu, Y., Wang, Q., Wang, Z., Wang, X., Ayyagari, B., Krishnan, S., Chudzik, M., & Lu, W. D. (2023). Bulk‐Switching Memristor‐Based Compute‐In‐Memory Module for Deep Neural Network Training. Advanced Materials, 35(46). https://doi.org/10.1002/adma.202305465

Vancouver

Wu Y, Wang Q, Wang Z, Wang X, Ayyagari B, Krishnan S, et al. Bulk‐Switching Memristor‐Based Compute‐In‐Memory Module for Deep Neural Network Training. Advanced Materials. 2023;35(46). doi:10.1002/adma.202305465

BibTeX
@article{Wu2023,
  author = {Wu, Yuting and Wang, Qiwen and Wang, Ziyu and Wang, Xinxin and Ayyagari, Buvna and Krishnan, Siddarth and Chudzik, Michael and Lu, Wei D},
  title = {Bulk‐Switching Memristor‐Based Compute‐In‐Memory Module for Deep Neural Network Training},
  journal = {Advanced Materials},
  year = {2023},
  volume = {35},
  number = {46},
  doi = {10.1002/adma.202305465}
}
Kostenlos starten →