EUVIMEDSkopri EUVIMED
Crossref

Fairness Concerns in App Reviews: A Study on AI-Based Mobile Apps

Ali Rezaei Nasab, Maedeh Dashti, Mojtaba Shahin, Mansooreh Zahedi, Hourieh Khalajzadeh, Chetan Arora, Peng Liang

ACM Transactions on Software Engineering and Methodology · 2025-02-28 · 34 (2) · 1-30

DOI: 10.1145/3690633 ↗

At a glance

Journal article in ACM Transactions on Software Engineering and Methodology, published online on 2025-01-25. Written by Ali Rezaei Nasab and 6 co-authors. Cited 12 times so far (Crossref); 110 references. Funded by: National Natural Science Foundation of China.

AstrattDettalji biblijografiċiCiteRelated studies

Abstract (original language)

Fairness is one of the socio-technical concerns that must be addressed in software systems. Considering the popularity of mobile software applications (apps) among a wide range of individuals worldwide, mobile apps with unfair behaviors and outcomes can affect a significant proportion of the global population, potentially more than any other type of software system. Users express a wide range of socio-technical concerns in mobile app reviews. This research aims to investigate fairness concerns raised in mobile app reviews. Our research focuses on AI-based mobile app reviews as the chance of unfair behaviors and outcomes in AI-based mobile apps may be higher than in non-AI-based apps. To this end, we first manually constructed a ground-truth dataset, including 1,132 fairness and 1,473 non-fairness reviews. Leveraging the ground-truth dataset, we developed and evaluated a set of machine learning and deep learning models that distinguish fairness reviews from non-fairness reviews. Our experiments show that our best-performing model can detect fairness reviews with a precision of 94%. We then applied the best-performing model on approximately 9.5M reviews collected from 108 AI-based apps and identified around 92K fairness reviews. Next, applying the K-means clustering technique to the 92K fairness reviews, followed by manual analysis, led to the identification of six distinct types of fairness concerns (e.g., “receiving different quality of features and services in different platforms and devices” and “lack of transparency and fairness in dealing with user-generated content” ). Finally, the manual analysis of 2,248 app owners’ responses to the fairness reviews identified six root causes (e.g., “copyright issues”) that app owners report to justify fairness concerns.

Funding

  • National Natural Science Foundation of China

Cite

APA

Nasab, A. R., Dashti, M., Shahin, M., Zahedi, M., Khalajzadeh, H., Arora, C., & Liang, P. (2025). Fairness Concerns in App Reviews: A Study on AI-Based Mobile Apps. ACM Transactions on Software Engineering and Methodology, 34(2), 1-30. https://doi.org/10.1145/3690633

Vancouver

Nasab AR, Dashti M, Shahin M, Zahedi M, Khalajzadeh H, Arora C, et al. Fairness Concerns in App Reviews: A Study on AI-Based Mobile Apps. ACM Transactions on Software Engineering and Methodology. 2025;34(2):1-30. doi:10.1145/3690633

BibTeX
@article{Nasab2025,
  author = {Nasab, Ali Rezaei and Dashti, Maedeh and Shahin, Mojtaba and Zahedi, Mansooreh and Khalajzadeh, Hourieh and Arora, Chetan and Liang, Peng},
  title = {Fairness Concerns in App Reviews: A Study on AI-Based Mobile Apps},
  journal = {ACM Transactions on Software Engineering and Methodology},
  year = {2025},
  volume = {34},
  number = {2},
  pages = {1--30},
  doi = {10.1145/3690633}
}
Start for free →