EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

ReFLAIR: Detecting Responsive Layout Reflow Issues using Multimodal Generative AI

Yirui He, Ziyao He, Syed Fatiul Huq, Sam Malek

Proceedings of the ACM on Software Engineering · 30.06.2026 · 3 (FSE) · 2905-2927

DOI: 10.1145/3808136 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Proceedings of the ACM on Software Engineering, online erschienen am 30.06.2026. Verfasst von Yirui He und 3 weiteren Autor:innen. Die Arbeit nennt 74 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: National Science Foundation.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

With over 60 percent of global Internet traffic originating from mobile devices, Responsive Web Design (RWD) has become essential for ensuring seamless user experiences across diverse screen sizes and resolutions. The Web Content Accessibility Guidelines require that both information and functionality remain accessible when reflow occurs. However, existing checkers or research tools have limitations: they either employ static analysis that fails to capture how content is actually displayed to users or focus on only one accessibility aspect, such as keyboard operation. Consequently, no prior study has comprehensively addressed both information and functionality loss during reflow for Graphical User Interfaces (GUIs). This paper introduces ReFLAIR (Reflow Fault Localization using AI-based Responsive analysis), a multi-modal generative AI–driven approach for dynamically detecting reflow issues that cause loss of information or functionality in the GUI. ReFLAIR systematically extracts informative and actionable widgets, compares their presence and behavior across original and reflowed layouts, and employs both scrolling and large language model–guided expansion to uncover hidden interface widgets. We evaluate ReFLAIR on a dataset of 24 diverse webpages drawn from popular sites, prior benchmarks, and newly released webpages. Results show that ReFLAIR outperforms five state-of-the-art techniques, achieving precision improvements of at least 20.49% and recall improvements of at least 55.40%, while maintaining reasonable computational and runtime cost. An ablation study confirms that dynamic exploration (i.e., scrolling and expansion) is essential for high accuracy. We evaluated scalability and generalizability by extending the dataset to 36 webpages, covering 28 domains and higher complexity, and experimenting with alternative models and viewports. The results reinforce ReFLAIR’s consistency across diverse subjects and configurations. In summary, our approach contributes to accessibility testing by providing an effective, scalable, and cost-efficient solution for identifying reflow issues in RWD.

Förderung

  • National Science Foundation

Zitieren

APA

He, Y., He, Z., Huq, S. F., & Malek, S. (2026). ReFLAIR: Detecting Responsive Layout Reflow Issues using Multimodal Generative AI. Proceedings of the ACM on Software Engineering, 3(FSE), 2905-2927. https://doi.org/10.1145/3808136

Vancouver

He Y, He Z, Huq SF, Malek S. ReFLAIR: Detecting Responsive Layout Reflow Issues using Multimodal Generative AI. Proceedings of the ACM on Software Engineering. 2026;3(FSE):2905-2927. doi:10.1145/3808136

BibTeX
@article{He2026,
  author = {He, Yirui and He, Ziyao and Huq, Syed Fatiul and Malek, Sam},
  title = {ReFLAIR: Detecting Responsive Layout Reflow Issues using Multimodal Generative AI},
  journal = {Proceedings of the ACM on Software Engineering},
  year = {2026},
  volume = {3},
  number = {FSE},
  pages = {2905--2927},
  doi = {10.1145/3808136}
}
Kostenlos starten →