Optimising Human-Machine Collaboration for Efficient High-Precision Information Extraction from Text Documents
ACM Journal on Responsible Computing · 2024
DOI: 10.1145/3652591 ↗Kurz gefasst
Fachartikel in der Zeitschrift ACM Journal on Responsible Computing, 2024. Verfasst von Bradley Butcher und 4 weiteren Autor:innen. Die Arbeit wurde bisher 7-mal zitiert (Crossref) und nennt 81 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: European Research Council, EPSRC, The Alan Turing Institute, and the Leverhulme Trust.
Abstract (Originalsprache)
From science to law enforcement, many research questions are answerable only by poring over a large amount of unstructured text documents. While people can extract information from such documents with high accuracy, this is often too time-consuming to be practical. On the other hand, automated approaches produce nearly-immediate results, but are not reliable enough for applications where near-perfect precision is essential. Motivated by two use cases from criminal justice, we consider the benefits and drawbacks of various human-only, human–machine, and machine-only approaches. Finding no tool well suited for our use cases, we develop a human-in-the-loop method for fast but accurate extraction of structured data from unstructured text. The tool is based on automated extraction followed by human validation, and is particularly useful in cases where purely manual extraction is not practical. Testing on three criminal justice datasets, we find that the combination of the computer speed and human understanding yields precision comparable to manual annotation while requiring only a fraction of time, and significantly outperforms the precision of all fully automated baselines.
Förderung
- European Research Council
- EPSRC
- The Alan Turing Institute, and the Leverhulme Trust
- Turing AI fellowship
- Leverhulme Trust via the Centre for the Future of Intelligence
Zitieren
APA
Butcher, B., Zilka, M., Hron, J., Cook, D., & Weller, A. (2024). Optimising Human-Machine Collaboration for Efficient High-Precision Information Extraction from Text Documents. ACM Journal on Responsible Computing. https://doi.org/10.1145/3652591
Vancouver
Butcher B, Zilka M, Hron J, Cook D, Weller A. Optimising Human-Machine Collaboration for Efficient High-Precision Information Extraction from Text Documents. ACM Journal on Responsible Computing. 2024. doi:10.1145/3652591
BibTeX
@article{Butcher2024,
author = {Butcher, Bradley and Zilka, Miri and Hron, Jiri and Cook, Darren and Weller, Adrian},
title = {Optimising Human-Machine Collaboration for Efficient High-Precision Information Extraction from Text Documents},
journal = {ACM Journal on Responsible Computing},
year = {2024},
doi = {10.1145/3652591}
}
EUVIMED