EUVIMEDEUVIMED entdecken
Crossref

Some Common Mistakes In IR Evaluation, And How They Can Be Avoided

Norbert Fuhr

ACM SIGIR Forum · 22.02.2018 · 51 (3) · 32-41

DOI: 10.1145/3190580.3190586 ↗

Kurz gefasst

Fachartikel in der Zeitschrift ACM SIGIR Forum, online erschienen am 22.02.2018. Verfasst von Norbert Fuhr. Die Arbeit wurde bisher 105-mal zitiert (Crossref) und nennt 22 Literaturangaben.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

This paper points out some mistakes that can be frequently found in IR publications: MRR and ERR violate basic requirements for a metric, MAP is based on unrealistic assumptions, the numbers shown overstate the precision of the result, relative improvements of arithmetic means are inappropriate, the simple holdout method yields unreliable results, hypotheses are often formulated after the experiment, significance tests frequently ignore the multiple comparisons problem, effect sizes are ignored, reproducibility of the experiments might be nearly impossible, and sometimes authors claim proof by experimentation.

Zitieren

APA

Fuhr, N. (2018). Some Common Mistakes In IR Evaluation, And How They Can Be Avoided. ACM SIGIR Forum, 51(3), 32-41. https://doi.org/10.1145/3190580.3190586

Vancouver

Fuhr N. Some Common Mistakes In IR Evaluation, And How They Can Be Avoided. ACM SIGIR Forum. 2018;51(3):32-41. doi:10.1145/3190580.3190586

BibTeX
@article{Fuhr2018,
  author = {Fuhr, Norbert},
  title = {Some Common Mistakes In IR Evaluation, And How They Can Be Avoided},
  journal = {ACM SIGIR Forum},
  year = {2018},
  volume = {51},
  number = {3},
  pages = {32--41},
  doi = {10.1145/3190580.3190586}
}
Kostenlos starten →