DocETL: Agentic Query Rewriting and Evaluation for Complex Document Processing
Proceedings of the VLDB Endowment · 2025-05 · 18 (9) · 3035-3048
DOI: 10.14778/3746405.3746426 ↗Kurz gefasst
Fachartikel in der Zeitschrift Proceedings of the VLDB Endowment, online erschienen am 03.09.2025. Verfasst von Shreya Shankar und 4 weiteren Autor:innen. Die Arbeit wurde bisher 43-mal zitiert (Crossref) und nennt 56 Literaturangaben.
Abstract (Originalsprache)
Analyzing unstructured data has been a persistent challenge in data processing. Recent proposals offer declarative frameworks for LLM-powered processing of unstructured data, but they typically execute user-specified operations as-is in a single LLM call—focusing on cost rather than accuracy. This is problematic for complex tasks, where even well-prompted LLMs can miss relevant information. For instance, reliably extracting all instances of a specific clause from legal documents often requires decomposing the task, the data, or both. We present DocETL, a system that optimizes complex document processing pipelines, while accounting for LLM shortcomings. DocETL offers a declarative interface for users to deine such pipelines and uses an agent-based approach to automatically optimize them, leveraging novel agent-based rewrites (that we call rewrite directives ), as well as an optimization and evaluation framework. We introduce (i) logical rewriting of pipelines, tailored for LLM-based tasks, (ii) an agent-guided plan evaluation mechanism, and (iii) an optimization algorithm that efficiently finds promising plans, considering the latencies of LLM execution. Across four real-world document processing tasks, DocETL improves accuracy by 21–80% over strong baselines. DocETL is open-source at docetl.org and, as of March 2025, has over 1.7k GitHub stars across diverse domains.
Zitieren
APA
Shankar, S., Chambers, T., Shah, T., Parameswaran, A. G., & Wu, E. (2025). DocETL: Agentic Query Rewriting and Evaluation for Complex Document Processing. Proceedings of the VLDB Endowment, 18(9), 3035-3048. https://doi.org/10.14778/3746405.3746426
Vancouver
Shankar S, Chambers T, Shah T, Parameswaran AG, Wu E. DocETL: Agentic Query Rewriting and Evaluation for Complex Document Processing. Proceedings of the VLDB Endowment. 2025;18(9):3035-3048. doi:10.14778/3746405.3746426
BibTeX
@article{Shankar2025,
author = {Shankar, Shreya and Chambers, Tristan and Shah, Tarak and Parameswaran, Aditya G. and Wu, Eugene},
title = {DocETL: Agentic Query Rewriting and Evaluation for Complex Document Processing},
journal = {Proceedings of the VLDB Endowment},
year = {2025},
volume = {18},
number = {9},
pages = {3035--3048},
doi = {10.14778/3746405.3746426}
}
EUVIMED