EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

Visual Speech Enhancement GANs Improve Noisy Telemedicine Consultations with Multimodal Lip Cues and Noisy Audio Streams

Adrian Hamilton, Gavin Morgan, Graham Brooks

International Journal of Information and Communication Technology Trends · 25.06.2026 · 6 (1) · 351-360

DOI: 10.66525/ijictt103 ↗

Kurz gefasst

Fachartikel in der Zeitschrift International Journal of Information and Communication Technology Trends, online erschienen am 25.06.2026. Verfasst von Adrian Hamilton und 2 weiteren Autor:innen. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

The rapid expansion of telemedicine has fundamentally altered the landscape of modern healthcare delivery, enabling remote consultations and expanding access to medical professionals. However, the efficacy of virtual medical consultations is frequently compromised by degraded audio quality stemming from unpredictable background noise in patient environments, low-bandwidth network transmission, and substandard consumer hardware. Traditional speech enhancement algorithms rely exclusively on acoustic features, which often fail in low signal-to-noise ratio conditions typical of telemedicine scenarios. To address this critical gap, this paper introduces a novel Visual Speech Enhancement Generative Adversarial Network designed specifically for noisy telemedicine consultation videos. By leveraging the multimodal nature of video consultations, the proposed architecture incorporates visual cues, specifically lip and facial movements, to guide the acoustic enhancement process. The system employs a dual-stream encoder to extract synchronized audio-visual features, followed by a generator network that reconstructs high-fidelity speech spectrograms, and a discriminator network that enforces realistic speech distributions. Comprehensive evaluations using a newly curated dataset of simulated telemedicine interactions demonstrate that the proposed multimodal approach significantly outperforms unimodal audio enhancement techniques across both objective acoustic metrics and subjective clinical intelligibility scores. The integration of visual data acts as a robust anchor during periods of extreme acoustic masking, preserving vital phonetic details required for accurate medical diagnosis. These findings highlight the transformative potential of audio-visual generative models in ensuring reliable, high-quality communication in remote healthcare settings.

Zitieren

APA

Hamilton, A., Morgan, G., & Brooks, G. (2026). Visual Speech Enhancement GANs Improve Noisy Telemedicine Consultations with Multimodal Lip Cues and Noisy Audio Streams. International Journal of Information and Communication Technology Trends, 6(1), 351-360. https://doi.org/10.66525/ijictt103

Vancouver

Hamilton A, Morgan G, Brooks G. Visual Speech Enhancement GANs Improve Noisy Telemedicine Consultations with Multimodal Lip Cues and Noisy Audio Streams. International Journal of Information and Communication Technology Trends. 2026;6(1):351-360. doi:10.66525/ijictt103

BibTeX
@article{Hamilton2026,
  author = {Hamilton, Adrian and Morgan, Gavin and Brooks, Graham},
  title = {Visual Speech Enhancement GANs Improve Noisy Telemedicine Consultations with Multimodal Lip Cues and Noisy Audio Streams},
  journal = {International Journal of Information and Communication Technology Trends},
  year = {2026},
  volume = {6},
  number = {1},
  pages = {351--360},
  doi = {10.66525/ijictt103}
}
Kostenlos starten →