An Accurate Deep Learning–Based System for Automatic Pill Identification: Model Development and Validation
Journal of Medical Internet Research · 2023
DOI: 10.2196/41043 ↗At a glance
Journal article in Journal of Medical Internet Research, 2023. Written by Junyeong Heo and 4 co-authors. Institution: Department of Mathematics, Korea University, Seoul, Republic of Korea. Major MEDLINE topics: Artificial Intelligence, Deep Learning. Cited 37 times so far (Crossref); 46 references. The full text is freely available in PubMed Central.
Abstract (original language)
Background
Medication errors account for a large proportion of all medical errors. In most homes, patients take a variety of medications for a long period. However, medication errors frequently occur because patients often throw away the containers of their medications.
Objective
We proposed a deep learning–based system for reducing medication errors by accurately identifying prescription pills. Given the pill images, our system located the pills in the respective pill databases in South Korea and the United States.
Methods
We organized the system into a pill recognition step and pill retrieval step, and we applied deep learning models to train not only images of the pill but also imprinted characters. In the pill recognition step, there are 3 modules that recognize the 3 features of pills and their imprints separately and correct the recognized imprint to fit the actual data. We adopted image classification and text detection models for the feature and imprint recognition modules, respectively. In the imprint correction module, we introduced a language model for the first time in the pill identification system and proposed a novel coordinate encoding technique for effective correction in the language model. We identified pills using similarity scores of pill characteristics with those in the database.
Results
We collected the open pill database from South Korea and the United States in May 2022. We used a total of 24,404 pill images in our experiments. The experimental results show that the predicted top-1 candidates achieve accuracy levels of 85.6% (South Korea) and 74.5% (United States) for the types of pills not trained on 2 different databases (South Korea and the United States). Furthermore, the predicted top-1 candidate accuracy of our system was 78% with consumer-granted images, which was achieved by training only 1 image per pill. The results demonstrate that our system could identify and retrieve new pills without additional model updates. Finally, we confirmed through an ablation study that the language model that we emphasized significantly improves the pill identification ability of the system.
Conclusions
Our study proposes the possibility of reducing medical errors by showing that the introduction of artificial intelligence can identify numerous pills with high precision in real time. Our study suggests that the proposed system can reduce patients’ misuse of medications and help medical staff focus on higher-level tasks by simplifying time-consuming lower-level tasks such as pill identification.
Subjects
MeSH terms (MEDLINE)
Author keywords
Authors and institutions
- Department of Mathematics, Korea University, Seoul, Republic of Korea.
- Visual Display Division, Samsung Electronics, Suwon, Republic of Korea.
- Department of Computer Science and Engineering, Korea University, Seoul, Republic of Korea.
- Department of Artificial Intelligence, Korea University, Seoul, Republic of Korea.
- Department of Artificial Intelligence, The Catholic University of Korea, Bucheon, Republic of Korea.
- Department of Data Science, The Catholic University of Korea, Bucheon, Republic of Korea.
ORCID profiles: Junyeong Heo, Youjin Kang, SangKeun Lee, Dong-Hwa Jeong, Kang-Min Kim
Cite
APA
Heo, J., Kang, Y., Lee, S., Jeong, D. H., & Kim, K. M. (2023). An Accurate Deep Learning–Based System for Automatic Pill Identification: Model Development and Validation. Journal of Medical Internet Research. https://doi.org/10.2196/41043
Vancouver
Heo J, Kang Y, Lee S, Jeong DH, Kim KM. An Accurate Deep Learning–Based System for Automatic Pill Identification: Model Development and Validation. Journal of Medical Internet Research. 2023. doi:10.2196/41043
BibTeX
@article{Heo2023,
author = {Heo, Junyeong and Kang, Youjin and Lee, SangKeun and Jeong, Dong-Hwa and Kim, Kang-Min},
title = {An Accurate Deep Learning–Based System for Automatic Pill Identification: Model Development and Validation},
journal = {Journal of Medical Internet Research},
year = {2023},
doi = {10.2196/41043}
}
EUVIMED