EUVIMEDEUVIMED entdecken
Open Access · CC BY-NC 4.0

Did I really agree to that?: making sense of policies you didn’t read with models that actually did

Shikha Soneji, Mitchell Hoesing, Sujay Koujalgi, Jonathan Dodge

Information Research an international electronic journal · 20.03.2026 · 31 (iConf) · 699-717

DOI: 10.47989/ir31iconf64157 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Information Research an international electronic journal, online erschienen am 20.03.2026. Verfasst von Shikha Soneji und 3 weiteren Autor:innen. Der Volltext ist frei zugänglich (Open Access, CC BY-NC 4.0).

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Introduction

While Privacy Policies and Terms of Service (ToS) are intended to inform users; they often overwhelm, mislead, and confuse in practice. This work investigates automated techniques for analyzing such legal documents, with the goal of supporting user comprehension and regulatory auditing.

Method

We use expert-driven annotations from Terms of Service; Didn’t Read (ToS;DR) to train classification models that assign case labels to individual sentences. We also develop a classifier to distinguish document types: Privacy Policy, ToS, or other legal text. Analysis. Models were evaluated using F1 score, and we compared traditional fine-tuned models (RoBERTa, PrivBERT) against GPT-4 Turbo. We then applied the best-performing models to real-world policies to uncover conceptual overlaps between Privacy Policies and ToS.

Results

Our case classifier achieved a 0.73 F1 score, while the document-type classifier reached 0.79. GPT-4 performed worse on case classification (0.58 F1). We found that GDPR-relevant clauses often appear in both Privacy Policies and ToS, blurring distinctions and raising risks for user misinterpretation and regulatory non-compliance.

Conclusion

By surfacing hidden structures and overlapping clauses, our system enhances transparency and supports digital literacy by increasing accessibility of complex documents. This work lays the foundation for tools that promote user agency and platform accountability.

Zitieren

APA

Soneji, S., Hoesing, M., Koujalgi, S., & Dodge, J. (2026). Did I really agree to that?: making sense of policies you didn’t read with models that actually did. Information Research an international electronic journal, 31(iConf), 699-717. https://doi.org/10.47989/ir31iconf64157

Vancouver

Soneji S, Hoesing M, Koujalgi S, Dodge J. Did I really agree to that?: making sense of policies you didn’t read with models that actually did. Information Research an international electronic journal. 2026;31(iConf):699-717. doi:10.47989/ir31iconf64157

BibTeX
@article{Soneji2026,
  author = {Soneji, Shikha and Hoesing, Mitchell and Koujalgi, Sujay and Dodge, Jonathan},
  title = {Did I really agree to that?: making sense of policies you didn’t read with models that actually did},
  journal = {Information Research an international electronic journal},
  year = {2026},
  volume = {31},
  number = {iConf},
  pages = {699--717},
  doi = {10.47989/ir31iconf64157}
}
Kostenlos starten →