EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

External benchmarking of lung ultrasound models for pneumothorax‐related signs: A manifest‐based multi‐source study

Takehiro Ishikawa

Clinical and Translational Discovery · 2026-06 · 6 (3)

DOI: 10.1002/ctd2.70152 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Clinical and Translational Discovery, online erschienen am 12.05.2026. Verfasst von Takehiro Ishikawa. Die Arbeit nennt 15 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Background and aims Reproducible external benchmarks for pneumothorax‐related lung ultrasound (LUS) AI are scarce, and binary lung‐sliding classification may obscure clinically important signs. We therefore developed a manifest‐based external benchmark and used it to test both cross‐domain generalisation and task validity.

Methods

We curated 280 clips from 190 publicly accessible LUS source videos and released a reconstruction manifest containing URLs, timestamps, crop coordinates, labels, and probe shape. Labels were normal lung sliding, absent lung sliding, lung point, and lung pulse. A previously published single‐site binary classifier was evaluated on this benchmark; challenge‐state analysis examined lung point and lung pulse using the predicted probability of absent sliding, P(absent).

Results

The single‐site comparator achieved Receiver Operating Characteristic–Area Under the Curve (ROC‐AUC) 0.9625 in‐domain but 0.7050 on the heterogeneous external benchmark; restricting external evaluation to linear clips still yielded ROC‐AUC 0.7212. In challenge‐state analysis, mean P(absent) ranked absent (0.504) > lung point (0.313) > normal (0.186) > lung pulse (0.143). Lung pulse differed from absent clips ( p = 0.000470) but not from normal clips ( p = 0.813), indicating that the binary model treated pulse as normal‐like despite absent sliding. Lung point differed from both absent ( p = 0.000468) and normal ( p = 0.000026), supporting its interpretation as an intermediate ambiguity state rather than a clean binary class.

Conclusion

A manifest‐based, multi‐source benchmark can support reproducible external evaluation without redistributing source videos. Binary lung‐sliding classification is an incomplete proxy for pneumothorax reasoning because it obscures blind‐spot and ambiguity states, such as lung pulse and lung point.

Zitieren

APA

Ishikawa, T. (2026). External benchmarking of lung ultrasound models for pneumothorax‐related signs: A manifest‐based multi‐source study. Clinical and Translational Discovery, 6(3). https://doi.org/10.1002/ctd2.70152

Vancouver

Ishikawa T. External benchmarking of lung ultrasound models for pneumothorax‐related signs: A manifest‐based multi‐source study. Clinical and Translational Discovery. 2026;6(3). doi:10.1002/ctd2.70152

BibTeX
@article{Ishikawa2026,
  author = {Ishikawa, Takehiro},
  title = {External benchmarking of lung ultrasound models for pneumothorax‐related signs: A manifest‐based multi‐source study},
  journal = {Clinical and Translational Discovery},
  year = {2026},
  volume = {6},
  number = {3},
  doi = {10.1002/ctd2.70152}
}
Kostenlos starten →