EUVIMEDDescubrir EUVIMED
Open access · CC BY 4.0

Multimodal Ontology: Semantic Substrate for Human Digital Twins in AI-Driven Metaverse Telerehabilitation and Immersive Medical Education

Oleksandr Palagin, Serhii Matvieishyn, Oleksii Bahatskyi, Dmytro Vakulenko, Kyrylo Malakhov

Metaverse · 2026-07-15 · 7 (3)

DOI: 10.54517/m8553 ↗

At a glance

Journal article in Metaverse, published online on 2026-07-15. Written by Oleksandr Palagin and 4 co-authors. The full text is freely available (open access, CC BY 4.0).

ResumenDatos bibliográficosCiteRelated studies

Abstract (original language)

The convergence of the metaverse with AI and digital twin technologies is reshaping healthcare delivery and medical education, yet Human Digital Twins (HDTs) – continuously updated virtual patient models – often lack a unifying semantic substrate capable of integrating heterogeneous multimodal evidence (sensor streams, clinical text, imaging, and interaction traces) into actionable meaning for perception, reasoning, and decision support. This viewpoint positions multimodal ontology as that substrate and as a prerequisite for robust AI-driven metaverse workflows in telerehabilitation and immersive medical education. Multimodal ontology is defined at a high level of abstraction and specified via a minimal formalization capturing its essential elements: domain concepts and relations, modality sets, semantic grounding operators that map raw observations to ontological entities, and state-update functions that maintain a temporally evolving HDT knowledge state. Building on recent HDT-enabled telerehabilitation work, the ontology layer is shown to operationalize core metaverse AI functions – environmental perception, semantic understanding, behavioral planning, and human–machine collaboration through interoperable representations, rule- and graph-based inference, and explainable agent behavior across clinician, patient, and intern interfaces. A testable research agenda is outlined for ontology-backed HDTs, emphasizing integration with large-scale multimodal models, validation in simulation and real-world deployments, and evaluation using semantic consistency, reasoning fidelity, user trust, and patient or intern outcomes. The specific contribution is not a new classifier, simulator, or network-optimization mechanism; rather, it is a formal semantic substrate that couples multimodal grounding, confidence-weighted fusion, uncertainty-aware HDT state evolution, and ontology-constrained interaction with large-scale foundation models.

Cite

APA

Palagin, O., Matvieishyn, S., Bahatskyi, O., Vakulenko, D., & Malakhov, K. (2026). Multimodal Ontology: Semantic Substrate for Human Digital Twins in AI-Driven Metaverse Telerehabilitation and Immersive Medical Education. Metaverse, 7(3). https://doi.org/10.54517/m8553

Vancouver

Palagin O, Matvieishyn S, Bahatskyi O, Vakulenko D, Malakhov K. Multimodal Ontology: Semantic Substrate for Human Digital Twins in AI-Driven Metaverse Telerehabilitation and Immersive Medical Education. Metaverse. 2026;7(3). doi:10.54517/m8553

BibTeX
@article{Palagin2026,
  author = {Palagin, Oleksandr and Matvieishyn, Serhii and Bahatskyi, Oleksii and Vakulenko, Dmytro and Malakhov, Kyrylo},
  title = {Multimodal Ontology: Semantic Substrate for Human Digital Twins in AI-Driven Metaverse Telerehabilitation and Immersive Medical Education},
  journal = {Metaverse},
  year = {2026},
  volume = {7},
  number = {3},
  doi = {10.54517/m8553}
}
Start for free →