EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

Unsupervised learning reveals interpretable latent representations for translucency perception

Chenxi Liao iD, Masataka Sawayama, Bei Xiao

PLOS Computational Biology · 08.02.2023 · 19 (2) · e1010878

DOI: 10.1371/journal.pcbi.1010878 ↗

Kurz gefasst

Fachartikel in der Zeitschrift PLOS Computational Biology, online erschienen am 08.02.2023. Verfasst von Chenxi Liao und 2 weiteren Autor:innen. Beteiligte Einrichtung: Department of Neuroscience, American University, Washington, D.C., District of Columbia, United States of America. Hauptthemen laut MEDLINE: Unsupervised Machine Learning, Lighting. Die Arbeit wurde bisher 29-mal zitiert (Crossref) und nennt 126 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: NEI NIH HHS.

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Humans constantly assess the appearance of materials to plan actions, such as stepping on icy roads without slipping. Visual inference of materials is important but challenging because a given material can appear dramatically different in various scenes. This problem especially stands out for translucent materials, whose appearance strongly depends on lighting, geometry, and viewpoint. Despite this, humans can still distinguish between different materials, and it remains unsolved how to systematically discover visual features pertinent to material inference from natural images. Here, we develop an unsupervised style-based image generation model to identify perceptually relevant dimensions for translucent material appearances from photographs. We find our model, with its layer-wise latent representation, can synthesize images of diverse and realistic materials. Importantly, without supervision, human-understandable scene attributes, including the object’s shape, material, and body color, spontaneously emerge in the model’s layer-wise latent space in a scale-specific manner. By embedding an image into the learned latent space, we can manipulate specific layers’ latent code to modify the appearance of the object in the image. Specifically, we find that manipulation on the early-layers (coarse spatial scale) transforms the object’s shape, while manipulation on the later-layers (fine spatial scale) modifies its body color. The middle-layers of the latent space selectively encode translucency features and manipulation of such layers coherently modifies the translucency appearance, without changing the object’s shape or body color. Moreover, we find the middle-layers of the latent space can successfully predict human translucency ratings, suggesting that translucent impressions are established in mid-to-low spatial scale features. This layer-wise latent representation allows us to systematically discover perceptually relevant image features for human translucency perception. Together, our findings reveal that learning the scale-specific statistical structure of natural images might be crucial for humans to efficiently represent material properties across contexts.

Schlagwörter

MeSH-Schlagwörter (MEDLINE)

Unsupervised Machine LearningLightingHumansAttitudeImaging, Three-DimensionalPerceptionVisual Perception

Autor:innen und Institutionen

  • Department of Neuroscience, American University, Washington, D.C., District of Columbia, United States of America.
  • Graduate School of Information Science and Technology, The University of Tokyo, Tokyo, Japan.
  • Department of Computer Science, American University, Washington, D.C., District of Columbia, United States of America.

ORCID-Profile: Chenxi Liao

Förderung

  • NEI NIH HHS

Zitieren

APA

Liao, C., Sawayama, M., & Xiao, B. (2023). Unsupervised learning reveals interpretable latent representations for translucency perception. PLOS Computational Biology, 19(2), e1010878. https://doi.org/10.1371/journal.pcbi.1010878

Vancouver

Liao C, Sawayama M, Xiao B. Unsupervised learning reveals interpretable latent representations for translucency perception. PLOS Computational Biology. 2023;19(2):e1010878. doi:10.1371/journal.pcbi.1010878

BibTeX
@article{Liao2023,
  author = {Liao, Chenxi and Sawayama, Masataka and Xiao, Bei},
  title = {Unsupervised learning reveals interpretable latent representations for translucency perception},
  journal = {PLOS Computational Biology},
  year = {2023},
  volume = {19},
  number = {2},
  pages = {e1010878},
  doi = {10.1371/journal.pcbi.1010878}
}
Kostenlos starten →