MechBERT: Language Models for Extracting Chemical and Property Relationships about Mechanical Stress and Strain
Journal of Chemical Information and Modeling · 24.02.2025 · 65 (4) · 1873-1888
DOI: 10.1021/acs.jcim.4c00857 ↗Kurz gefasst
Fachartikel in der Zeitschrift Journal of Chemical Information and Modeling, online erschienen am 31.01.2025. Verfasst von Pankaj Kumar und 2 weiteren Autor:innen. Beteiligte Einrichtung: Cavendish Laboratory, Department of Physics, University of Cambridge, J. J. Thomson Avenue, Cambridge CB3 0HE. U.K. Hauptthemen laut MEDLINE: Stress, Mechanical, Natural Language Processing. Die Arbeit wurde bisher 12-mal zitiert (Crossref) und nennt 48 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: Royal Academy of Engineering, Basic Energy Sciences, ISIS Neutron and Muon Source.
Abstract (Originalsprache)
Language models are transforming materials-aware natural-language processing by enabling the extraction of dynamic, context-rich information from unstructured text, thus, moving beyond the limitations of traditional information-extraction methods. Moreover, small language models are on the rise because some of them can perform better than large language models (LLMs) when given domain-specific question-answer tasks, especially about an application area that relies on a highly specialized vernacular, such as materials science. We therefore present a new class of MechBERT language models for understanding mechanical stress and strain in materials. These employ Bidirectional Encoder Representations for transformer (BERT) architectures. We showcase four MechBERT models, all of which were pretrained on a corpus of documents that are textually rich in chemicals and their stress-strain properties and were fine-tuned on question-answering tasks. We evaluated the level of performance of our models on domain-specific as well as general English-language question-answer tasks and also explored the influence of the size and type of BERT architectures on model performance. We find that our MechBERT models outperform BERT-based models of the same size and maintain relevancy better than much larger BERT-based models when tasked with domain-specific question-answering tasks within the stress-strain engineering sector. These small language models also enable much faster processing and require a much smaller fraction of data to pretrain them, affording them greater operational efficiency and energy sustainability than LLMs.
Schlagwörter
MeSH-Schlagwörter (MEDLINE)
Autor:innen und Institutionen
- Cavendish Laboratory, Department of Physics, University of Cambridge, J. J. Thomson Avenue, Cambridge CB3 0HE. U.K.
- ISIS Neutron and Muon Source, STFC Rutherford Appleton Laboratory, Harwell Science and Innovation Campus, Didcot OX11 0QX, U.K.
- Research Complex at Harwell, Rutherford Appleton Laboratory, Harwell Science and Innovation Campus, Didcot OX11 0FA, U.K.
ORCID-Profile: Jacqueline M Cole
Förderung
- Royal Academy of Engineering
- Basic Energy Sciences
- ISIS Neutron and Muon Source
Zitieren
APA
Kumar, P., Kabra, S., & Cole, J. M. (2025). MechBERT: Language Models for Extracting Chemical and Property Relationships about Mechanical Stress and Strain. Journal of Chemical Information and Modeling, 65(4), 1873-1888. https://doi.org/10.1021/acs.jcim.4c00857
Vancouver
Kumar P, Kabra S, Cole JM. MechBERT: Language Models for Extracting Chemical and Property Relationships about Mechanical Stress and Strain. Journal of Chemical Information and Modeling. 2025;65(4):1873-1888. doi:10.1021/acs.jcim.4c00857
BibTeX
@article{Kumar2025,
author = {Kumar, Pankaj and Kabra, Saurabh and Cole, Jacqueline M},
title = {MechBERT: Language Models for Extracting Chemical and Property Relationships about Mechanical Stress and Strain},
journal = {Journal of Chemical Information and Modeling},
year = {2025},
volume = {65},
number = {4},
pages = {1873--1888},
doi = {10.1021/acs.jcim.4c00857}
}
EUVIMED