OpticalBERT and OpticalTable-SQA: Text- and Table-Based Language Models for the Optical-Materials Domain
Journal of Chemical Information and Modeling · 2023
DOI: 10.1021/acs.jcim.2c01259 ↗Kurz gefasst
Fachartikel in der Zeitschrift Journal of Chemical Information and Modeling, 2023. Verfasst von Jiuyang Zhao und 2 weiteren Autor:innen. Beteiligte Einrichtung: Cavendish Laboratory, University of Cambridge, J. J. Thomson Avenue, Cambridge CB3 0HE, U.K. Hauptthemen laut MEDLINE: Data Mining, Electric Power Supplies. Die Arbeit wurde bisher 52-mal zitiert (Crossref) und nennt 44 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: Royal Academy of Engineering, Christ's College, University of Cambridge, China Scholarship Council.
Abstract (Originalsprache)
Text mining in the optical-materials domain is becoming increasingly important as the number of scientific publications in this area grows rapidly. Language models such as Bidirectional Encoder Representations from Transformers (BERT) have opened up a new era and brought a significant boost to state-of-the-art natural-language-processing (NLP) tasks. In this paper, we present two “materials-aware” text-based language models for optical research, OpticalBERT and OpticalPureBERT, which are trained on a large corpus of scientific literature in the optical-materials domain. These two models outperform BERT and previous state-of-the-art models in a variety of text-mining tasks about optical materials. We also release the first “materials-aware” table-based language model, OpticalTable-SQA. This is a querying facility that solicits answers to questions about optical materials using tabular information that pertains to this scientific domain. The OpticalTable-SQA model was realized by fine-tuning the Tapas-SQA model using a manually annotated OpticalTableQA data set which was curated specifically for this work. While preserving its sequential question-answering performance on general tables, the OpticalTable-SQA model significantly outperforms Tapas-SQA on optical-materials-related tables. All models and data sets are available to the optical-materials-science community.
Schlagwörter
MeSH-Schlagwörter (MEDLINE)
Autor:innen und Institutionen
- Cavendish Laboratory, University of Cambridge, J. J. Thomson Avenue, Cambridge CB3 0HE, U.K.
- ISIS Neutron and Muon Source, Rutherford Appleton Laboratory, Harwell Science and Innovation Campus, Didcot, Oxfordshire OX11 0QX, U.K.
ORCID-Profile: Jiuyang Zhao, Shu Huang, Jacqueline M Cole
Förderung
- Royal Academy of Engineering
- Christ's College, University of Cambridge
- China Scholarship Council
- Cambridge Trust
Zitieren
APA
Zhao, J., Huang, S., & Cole, J. M. (2023). OpticalBERT and OpticalTable-SQA: Text- and Table-Based Language Models for the Optical-Materials Domain. Journal of Chemical Information and Modeling. https://doi.org/10.1021/acs.jcim.2c01259
Vancouver
Zhao J, Huang S, Cole JM. OpticalBERT and OpticalTable-SQA: Text- and Table-Based Language Models for the Optical-Materials Domain. Journal of Chemical Information and Modeling. 2023. doi:10.1021/acs.jcim.2c01259
BibTeX
@article{Zhao2023,
author = {Zhao, Jiuyang and Huang, Shu and Cole, Jacqueline M},
title = {OpticalBERT and OpticalTable-SQA: Text- and Table-Based Language Models for the Optical-Materials Domain},
journal = {Journal of Chemical Information and Modeling},
year = {2023},
doi = {10.1021/acs.jcim.2c01259}
}
EUVIMED