EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

Rethinking multimodal corpora from the perspective of Peircean semiotics

Tuomo Hiippala

Frontiers in Communication · 12.02.2024 · 9

DOI: 10.3389/fcomm.2024.1337434 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Frontiers in Communication, online erschienen am 12.02.2024. Verfasst von Tuomo Hiippala. Die Arbeit wurde bisher 5-mal zitiert (Crossref) und nennt 58 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: Helsingin Yliopisto.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

This article discusses annotating and querying multimodal corpora from the perspective of Peircean semiotics. Corpora have had a significant impact on empirical research in the field of linguistics and are increasingly considered essential for multimodality research as well. I argue that Peircean semiotics can be used to gain a deeper understanding of multimodal corpora and rethink the way we work with them. I demonstrate the proposed approach in an empirical study, which uses Peircean semiotics to guide the process of querying multimodal corpora using computer vision and vector-based information retrieval. The results show that computer vision algorithms are restricted to particular domains of experience, which may be circumscribed using Peirce's theory of semiotics. However, the applicability of such algorithms may be extended using annotations, which capture aspects of meaning-making that remain beyond algorithms. Overall, the results suggest that the process of building and analysing multimodal corpora should be actively theorized in order to identify new ways of working with the information stored in them, particularly in terms of dividing the annotation tasks between humans and algorithms.

Förderung

  • Helsingin Yliopisto

Zitieren

APA

Hiippala, T. (2024). Rethinking multimodal corpora from the perspective of Peircean semiotics. Frontiers in Communication, 9. https://doi.org/10.3389/fcomm.2024.1337434

Vancouver

Hiippala T. Rethinking multimodal corpora from the perspective of Peircean semiotics. Frontiers in Communication. 2024;9. doi:10.3389/fcomm.2024.1337434

BibTeX
@article{Hiippala2024,
  author = {Hiippala, Tuomo},
  title = {Rethinking multimodal corpora from the perspective of Peircean semiotics},
  journal = {Frontiers in Communication},
  year = {2024},
  volume = {9},
  doi = {10.3389/fcomm.2024.1337434}
}
Kostenlos starten →