External benchmarking of lung ultrasound models for pneumothorax‐related signs: A manifest‐based multi‐source study
Clinical and Translational Discovery · 2026-06 · 6 (3)
DOI: 10.1002/ctd2.70152 ↗Kurz gefasst
Fachartikel in der Zeitschrift Clinical and Translational Discovery, online erschienen am 12.05.2026. Verfasst von Takehiro Ishikawa. Die Arbeit nennt 15 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).
Abstract (Originalsprache)
Background and aims Reproducible external benchmarks for pneumothorax‐related lung ultrasound (LUS) AI are scarce, and binary lung‐sliding classification may obscure clinically important signs. We therefore developed a manifest‐based external benchmark and used it to test both cross‐domain generalisation and task validity.
Methods
We curated 280 clips from 190 publicly accessible LUS source videos and released a reconstruction manifest containing URLs, timestamps, crop coordinates, labels, and probe shape. Labels were normal lung sliding, absent lung sliding, lung point, and lung pulse. A previously published single‐site binary classifier was evaluated on this benchmark; challenge‐state analysis examined lung point and lung pulse using the predicted probability of absent sliding, P(absent).
Results
The single‐site comparator achieved Receiver Operating Characteristic–Area Under the Curve (ROC‐AUC) 0.9625 in‐domain but 0.7050 on the heterogeneous external benchmark; restricting external evaluation to linear clips still yielded ROC‐AUC 0.7212. In challenge‐state analysis, mean P(absent) ranked absent (0.504) > lung point (0.313) > normal (0.186) > lung pulse (0.143). Lung pulse differed from absent clips ( p = 0.000470) but not from normal clips ( p = 0.813), indicating that the binary model treated pulse as normal‐like despite absent sliding. Lung point differed from both absent ( p = 0.000468) and normal ( p = 0.000026), supporting its interpretation as an intermediate ambiguity state rather than a clean binary class.
Conclusion
A manifest‐based, multi‐source benchmark can support reproducible external evaluation without redistributing source videos. Binary lung‐sliding classification is an incomplete proxy for pneumothorax reasoning because it obscures blind‐spot and ambiguity states, such as lung pulse and lung point.
Zitieren
APA
Ishikawa, T. (2026). External benchmarking of lung ultrasound models for pneumothorax‐related signs: A manifest‐based multi‐source study. Clinical and Translational Discovery, 6(3). https://doi.org/10.1002/ctd2.70152
Vancouver
Ishikawa T. External benchmarking of lung ultrasound models for pneumothorax‐related signs: A manifest‐based multi‐source study. Clinical and Translational Discovery. 2026;6(3). doi:10.1002/ctd2.70152
BibTeX
@article{Ishikawa2026,
author = {Ishikawa, Takehiro},
title = {External benchmarking of lung ultrasound models for pneumothorax‐related signs: A manifest‐based multi‐source study},
journal = {Clinical and Translational Discovery},
year = {2026},
volume = {6},
number = {3},
doi = {10.1002/ctd2.70152}
}
EUVIMED