EUVIMEDEUVIMED entdecken
Crossref

The Neural Hype and Comparisons Against Weak Baselines

Jimmy Lin

ACM SIGIR Forum · 17.01.2019 · 52 (2) · 40-51

DOI: 10.1145/3308774.3308781 ↗

Kurz gefasst

Fachartikel in der Zeitschrift ACM SIGIR Forum, online erschienen am 17.01.2019. Verfasst von Jimmy Lin. Die Arbeit wurde bisher 97-mal zitiert (Crossref) und nennt 17 Literaturangaben.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Recently, the machine learning community paused in a moment of self-reflection. In a widelydiscussed paper at ICLR 2018, Sculley et al. [13] wrote: "We observe that the rate of empirical advancement may not have been matched by consistent increase in the level of empirical rigor across the field as a whole." Their primary complaint is the development of a "research and publication culture that emphasizes wins" (emphasis in original), which typically means "demonstrating that a new method beats previous methods on a given task or benchmark". An apt description might be "leaderboard chasing"-and for many vision and NLP tasks, this isn't a metaphor. There are literally centralized leaderboards1 that track incremental progress, down to the fifth decimal point, some persisting over years, accumulating dozens of entries. Sculley et al. remind us that "the goal of science is not wins, but knowledge". The structure of the scientific enterprise today (pressure to publish, pace of progress, etc.) means that "winning" and "doing good science" are often not fully aligned. To wit, they cite a number of papers showing that recent advances in neural networks could very well be attributed to mundane issues like better hyperparameter optimization. Many results can't be reproduced, and some observed improvements might just be noise.

Zitieren

APA

Lin, J. (2019). The Neural Hype and Comparisons Against Weak Baselines. ACM SIGIR Forum, 52(2), 40-51. https://doi.org/10.1145/3308774.3308781

Vancouver

Lin J. The Neural Hype and Comparisons Against Weak Baselines. ACM SIGIR Forum. 2019;52(2):40-51. doi:10.1145/3308774.3308781

BibTeX
@article{Lin2019,
  author = {Lin, Jimmy},
  title = {The Neural Hype and Comparisons Against Weak Baselines},
  journal = {ACM SIGIR Forum},
  year = {2019},
  volume = {52},
  number = {2},
  pages = {40--51},
  doi = {10.1145/3308774.3308781}
}
Kostenlos starten →