EUVIMEDEUVIMED entdecken
Crossref

Measuring Source Fidelity, Exculpatory Recall, and Generative Suspicion in AI-Drafted Police Reports

Aizierjiang Aiersilan, Yeershanati Wuernikebai

Proceedings of the AAAI Symposium Series · 06.11.2026 · 10 (1) · 226-236

DOI: 10.1609/aaaiss.v10i1.43410 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Proceedings of the AAAI Symposium Series, online erschienen am 06.11.2026. Verfasst von Aizierjiang Aiersilan und 1 weiteren Autor:innen.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

California Senate Bill 524, codified at Penal Code § 13663 and effective 1 January 2026, is the most prescriptive state regulation to date of the artificial-intelligence systems that draft official law-enforcement reports. It compels disclosure, officer certification, retention of the first AI draft, and an audit trail, yet it says nothing about the substantive quality of the draft it makes discoverable, and the empirical record to date consists of a field trial of writing speed, officers' own perceptions, and human-rater quality scales. We propose FACETS-PR (Fidelity, Asymmetric exculpatory recall, Cognitive disengagement, Equity, Traceability, Suspicion inflation, for Police Reports), a six-axis framework of sixteen metrics, each treated as a measurement model whose construct-validity claim is stated explicitly rather than assumed. Two components are new: a computational form for the doctrinal notion of generative suspicion, and a paired-dialect probe, adapted from the matched-guise paradigm, that pairs Standard American English transcripts with African American English renderings of the same propositional content. A vendor-independent protocol accompanies the framework, running end to end on open-weight models over a synthetic, ground-truth-annotated body-worn camera corpus stratified by conditions that vendor documentation identifies as degrading draft quality. Instantiating it on four open-weight models yields five findings and two surprises. Quotation-marked retention of evidentiary utterances falls far below one half for every model, with large effect sizes, although a per-utterance audit attributes more of the shortfall, pooled across the panel, to verbatim reproduction stripped of its quotation marks than to paraphrase. No model emits the statutory disclosure string by default. Inculpatory content is recalled more reliably than exculpatory content in two of four models, an asymmetry that survives a direct test against the competing explanation that it merely tracks ground-truth set sizes. Sentences asserting legal conclusions that no single transcript turn entails appear in every model, at a rate that is reliably above zero when the panel is pooled but that no individual model's test resolves after correction for multiple comparisons. Officer-role reviewers leave roughly one deliberately impossible sentence in five standing. Against prediction, the drafts hedge rather than harden the transcript's epistemic stance, and we detect no dialect disparity on any suspicion-axis metric, nor on the recall asymmetry. The panel is deliberately modest, comprising quantised three- to eight-billion-parameter checkpoints rather than the frontier-scale proprietary systems on which deployed products are built, so these magnitudes characterise the instrument on a designed stress test rather than any commercial tool. Verification, disclosure, and accountability cannot be exercised over what has never been characterised; FACETS-PR is offered as one candidate instrument, and its reference implementation is released with this manuscript.

Zitieren

APA

Aiersilan, A., & Wuernikebai, Y. (2026). Measuring Source Fidelity, Exculpatory Recall, and Generative Suspicion in AI-Drafted Police Reports. Proceedings of the AAAI Symposium Series, 10(1), 226-236. https://doi.org/10.1609/aaaiss.v10i1.43410

Vancouver

Aiersilan A, Wuernikebai Y. Measuring Source Fidelity, Exculpatory Recall, and Generative Suspicion in AI-Drafted Police Reports. Proceedings of the AAAI Symposium Series. 2026;10(1):226-236. doi:10.1609/aaaiss.v10i1.43410

BibTeX
@article{Aiersilan2026,
  author = {Aiersilan, Aizierjiang and Wuernikebai, Yeershanati},
  title = {Measuring Source Fidelity, Exculpatory Recall, and Generative Suspicion in AI-Drafted Police Reports},
  journal = {Proceedings of the AAAI Symposium Series},
  year = {2026},
  volume = {10},
  number = {1},
  pages = {226--236},
  doi = {10.1609/aaaiss.v10i1.43410}
}
Kostenlos starten →