Ultrasound Image-Based Average $Q$-Learning Control of Magnetic Microrobots
IEEE Transactions on Robotics · 2025 · 41 · 1728-1741
DOI: 10.1109/tro.2025.3543261 ↗Kurz gefasst
Fachartikel in der Zeitschrift IEEE Transactions on Robotics, 2025. Verfasst von Jia Liu und 5 weiteren Autor:innen. Die Arbeit wurde bisher 18-mal zitiert (Crossref) und nennt 39 Literaturangaben. Gefördert von: National Key Research and Development Project, National Natural Science Foundation of China, Shenzhen Science and Technology Program.
Abstract (Originalsprache)
Magnetic microrobots have garnered significant attention and hold great potential for biomedical research applications. However, achieving precise manipulation in vivo poses significant challenges, particularly in medical image-based real-time feedback control, because it is difficult for a visual camera to track the motion of magnetic microrobots inside the body in biomedical applications. To realize the precise control of magnetic microrobots, it is also necessary to design and implement a simple and powerful control method. This approach allows for avoiding resource-intensive and complex control strategies. In this article, we present a learning-based real-time control method utilizing ultrasound images. Inspired by the ADboost concept, we use a reinforcement learning approach to integrate two simple control methods: a proportional-integral-derivative controller and a guiding vector field controller. We develop a novel $Q$-learning method called average $Q$-learning that incorporates average operation and $n$-step bootstraps. Its primary objective is to dynamically adjust the outputs of the different simple controllers. While each controller individually offers a straightforward solution, their integration contributes to a powerful control approach. To demonstrate its scalability, a nonsmooth path is utilized to investigate the integration performance of three simple controllers. In addition, we enhance a classic segmentation module, U-net, by incorporating an atrous spatial pyramid pooling module. To validate the effectiveness of the proposed control method, we conduct simulations and experiments using various planar paths. The quantitative analysis of the results demonstrates the efficacy of our approach in achieving precise manipulation, leveraging real-time control based on medical images for magnetic microrobots. Overall, this study provides a preliminary investigation into the field of medical image-based precise manipulation of magnetic microrobots in vivo applications.
Förderung
- National Key Research and Development Project
- National Natural Science Foundation of China
- Shenzhen Science and Technology Program
- Basic and Applied Basic Research Foundation of Guangdong Province
- SIAT-CUHK Joint Laboratory of Robotics and Intelligent Systems
Zitieren
APA
Liu, J., Ma, G., Fu, S., Huang, C., Wu, X., & Xu, T. (2025). Ultrasound Image-Based Average $Q$-Learning Control of Magnetic Microrobots. IEEE Transactions on Robotics, 41, 1728-1741. https://doi.org/10.1109/tro.2025.3543261
Vancouver
Liu J, Ma G, Fu S, Huang C, Wu X, Xu T. Ultrasound Image-Based Average $Q$-Learning Control of Magnetic Microrobots. IEEE Transactions on Robotics. 2025;41:1728-1741. doi:10.1109/tro.2025.3543261
BibTeX
@article{Liu2025,
author = {Liu, Jia and Ma, Guoyao and Fu, Shixiong and Huang, Chenyang and Wu, Xinyu and Xu, Tiantian},
title = {Ultrasound Image-Based Average $Q$-Learning Control of Magnetic Microrobots},
journal = {IEEE Transactions on Robotics},
year = {2025},
volume = {41},
pages = {1728--1741},
doi = {10.1109/tro.2025.3543261}
}
EUVIMED