EUVIMEDEUVIMED entdecken
Crossref

AlignDC: A Scalable and Reliable Ranking Algorithm for Noisy Crowdsourced Pairwise Comparisons

Ming-Hung Wang, Han-Ping Wu, Yu-Yu Chang, Hsiu-Ling Chu, Wei-Bin Lee

IEEE Transactions on Reliability · 2026 · 75 · 2319-2333

DOI: 10.1109/tr.2026.3705142 ↗

Kurz gefasst

Fachartikel in der Zeitschrift IEEE Transactions on Reliability, 2026. Verfasst von Ming-Hung Wang und 4 weiteren Autor:innen. Die Arbeit nennt 33 Literaturangaben. Gefördert von: National Science and Technology Council of Taiwan, Advanced Institute of Manufacturing with High-tech Innovations, Featured Areas Research Center Program.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

This article introduces AlignDC, a scalable and reliable algorithm for ranking items from noisy crowdsourced pairwise comparisons. By leveraging a divide-and-conquer strategy, AlignDC improves efficiency while maintaining robustness in large-scale datasets. Aggregating noisy crowdsourced pairwise comparisons for reliable ranking remains challenging due to scalability limits and substantial label noise. Existing divide-and-conquer methods reduce comparison cost but suffer from pivot misalignment in random groupings. The core innovation of AlignDC lies in reimagining the conquer phase through a principled global pivot alignment mechanism that fundamentally resolves local-global misalignment, substantially reducing variance in interpolated scores and achieving superior ranking accuracy under high label noise without additional computational or comparison cost. Extensive experiments demonstrate that AlignDC achieves an average accuracy improvement of 10.04%–23.70% over prior methods across various dataset sizes. Notably, under resource constraints, it delivers over 35% improvement for item sizes of 200. On two real-world datasets—text readability (490 items) and image comparison (9150 items)—AlignDC achieves average accuracy improvements of 13.26% and 8.99%, with peak accuracies of 96.13% (nearly reaching the baseline) and 86.08%, while requiring only 42.48%–73.49% and 75.71%–93.77% of comparisons used by the full pairwise baseline, respectively. The algorithm exhibits strong performance in both noisy and clean data settings, ensuring scalability for large datasets and high-volume comparisons. These advancements position AlignDC as a robust and efficient solution for real-world ranking problems where data quality and computational efficiency are critical.

Förderung

  • National Science and Technology Council of Taiwan
  • Advanced Institute of Manufacturing with High-tech Innovations
  • Featured Areas Research Center Program
  • Ministry of Education

Zitieren

APA

Wang, M., Wu, H., Chang, Y., Chu, H., & Lee, W. (2026). AlignDC: A Scalable and Reliable Ranking Algorithm for Noisy Crowdsourced Pairwise Comparisons. IEEE Transactions on Reliability, 75, 2319-2333. https://doi.org/10.1109/tr.2026.3705142

Vancouver

Wang M, Wu H, Chang Y, Chu H, Lee W. AlignDC: A Scalable and Reliable Ranking Algorithm for Noisy Crowdsourced Pairwise Comparisons. IEEE Transactions on Reliability. 2026;75:2319-2333. doi:10.1109/tr.2026.3705142

BibTeX
@article{Wang2026,
  author = {Wang, Ming-Hung and Wu, Han-Ping and Chang, Yu-Yu and Chu, Hsiu-Ling and Lee, Wei-Bin},
  title = {AlignDC: A Scalable and Reliable Ranking Algorithm for Noisy Crowdsourced Pairwise Comparisons},
  journal = {IEEE Transactions on Reliability},
  year = {2026},
  volume = {75},
  pages = {2319--2333},
  doi = {10.1109/tr.2026.3705142}
}
Kostenlos starten →