EUVIMEDEUVIMED entdecken
Open Access · CC BY-NC 4.0

Self-supervised learning with multimodal remote sensed maps for seafloor visual class inference

Cailei Liang, Jose Cappelletto, Miquel Massot-Campos, Adrian Bodenmann, Veerle AI Huvenne, Catherine Wardell, Brian J. Bett, Darryl Newborough, Blair Thornton

The International Journal of Robotics Research · 2025-12 · 44 (14) · 2340-2364

DOI: 10.1177/02783649251343640 ↗

Kurz gefasst

Fachartikel in der Zeitschrift The International Journal of Robotics Research, online erschienen am 27.05.2025. Verfasst von Cailei Liang und 8 weiteren Autor:innen. Die Arbeit wurde bisher 4-mal zitiert (Crossref) und nennt 87 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY-NC 4.0). Gefördert von: Kakenhi Grant Aid B of the Japan Society Promotion of Science.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Seafloor surveys often gather multiple modes of remote sensed mapping and sampling data to infer kilo- to mega-hectare scale seafloor habitat distributions. However, efforts to extract information from multimodal data are complicated by inconsistencies between measurement modes (e.g., resolution, positional offsets, geometric distortions) and different acquisition periods for dynamically changing environments. In this study, we investigate the use of location information during multimodal feature learning and its impact on habitat classification. Experiments on multimodal datasets gathered from three Marine Protected Areas (MPAs) showed improved robustness and performance when using location-based regularisation terms compared to equivalent autoencoder-based and contrastive self-supervised feature learners. Location-guiding improved F1 scores by 7.7% for autoencoder-based and 28.8% for contrastive feature learners averaged across 78 experiments on datasets spanning three distinct sites and 18 data modes. Location-guiding enhances performance when combining multimodal data, increasing F1 scores by an average of 8.8% and 37.8% compared to the best-performing individual mode being combined for autoencoder-based and contrastive self-supervised models, respectively. Performance gains are maintained over a large range of location-guiding distance hyperparameters, where improvements of 5.3% and 29.4% are achieved on average over an order-of-magnitude range of hyperparameters for the autoencoder and contrastive learners, respectively, both comparing favourably with optimally tuned conditions. Location-guiding also exhibits robustness to position inconsistencies between combined data modes, still achieving an average of 3.0% and 30.4% increase in performance compared to equivalent feature learners without location regularisation when position offsets of up to 10 m are artificially introduced to the remote sensed data. Our results show that the classifier used to delineate the learned feature spaces has less impact on performance than the feature learner, with probabilistic classifiers averaging 3.4% higher F1 scores than non-probabilistic classifiers.

Förderung

  • Kakenhi Grant Aid B of the Japan Society Promotion of Science

Zitieren

APA

Liang, C., Cappelletto, J., Massot-Campos, M., Bodenmann, A., Huvenne, V. A., Wardell, C., Bett, B. J., Newborough, D., & Thornton, B. (2025). Self-supervised learning with multimodal remote sensed maps for seafloor visual class inference. The International Journal of Robotics Research, 44(14), 2340-2364. https://doi.org/10.1177/02783649251343640

Vancouver

Liang C, Cappelletto J, Massot-Campos M, Bodenmann A, Huvenne VA, Wardell C, et al. Self-supervised learning with multimodal remote sensed maps for seafloor visual class inference. The International Journal of Robotics Research. 2025;44(14):2340-2364. doi:10.1177/02783649251343640

BibTeX
@article{Liang2025,
  author = {Liang, Cailei and Cappelletto, Jose and Massot-Campos, Miquel and Bodenmann, Adrian and Huvenne, Veerle AI and Wardell, Catherine and Bett, Brian J. and Newborough, Darryl and Thornton, Blair},
  title = {Self-supervised learning with multimodal remote sensed maps for seafloor visual class inference},
  journal = {The International Journal of Robotics Research},
  year = {2025},
  volume = {44},
  number = {14},
  pages = {2340--2364},
  doi = {10.1177/02783649251343640}
}
Kostenlos starten →