Inference‐Sufficient Representations for High‐Throughput Measurement: Lessons from Lossless Compression Benchmarks in 4D‐STEM
Advanced Computing · 2026-12 · 1 (1)
DOI: 10.1002/adco.70015 ↗Kurz gefasst
Fachartikel in der Zeitschrift Advanced Computing, online erschienen am 05.11.2026. Verfasst von Ondrej Dyck und 5 weiteren Autor:innen. Die Arbeit nennt 36 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: Basic Energy Sciences, Laboratory Directed Research and Development.
Abstract (Originalsprache)
Four‐dimensional scanning transmission electron microscopy (4D‐STEM) generates datasets whose volume already outpaces practical storage, transfer, and analysis. We benchmark 13 off‐the‐shelf lossless compression implementations across five datasets and find that the Blosc family substantially outperforms the compressors built into HDF5, achieving comparable compression ratios at far higher throughput. The broader conclusion is that this route is closing. Lossless compression is bounded by the entropy of the source, and on a simulated dataset where that bound can be computed exactly, the best implementation already comes within a factor of 2.5. A better implementation would recover a small multiple, not the orders of magnitude that rising detector rates demand. What must be decided instead is which data to keep and which to discard. Researchers already make this judgement routinely and often intuitively; it will have to be made explicit as agentic systems take on data collection and analysis. We argue for inference‐sufficient representations: a record suffices when it retains what an intended inference requires, not when it is complete. We distinguish coarsening, which cannot be false, from interpretation, which can, and show that event‐based detectors already perform the latter. We suggest design principles for deciding what a given inference requires.
Förderung
- Basic Energy Sciences
- Laboratory Directed Research and Development
Zitieren
APA
Dyck, O., Lupini, A. R., Borisevich, A., Chi, M., Vasudevan, R. K., & Jesse, S. (2026). Inference‐Sufficient Representations for High‐Throughput Measurement: Lessons from Lossless Compression Benchmarks in 4D‐STEM. Advanced Computing, 1(1). https://doi.org/10.1002/adco.70015
Vancouver
Dyck O, Lupini AR, Borisevich A, Chi M, Vasudevan RK, Jesse S. Inference‐Sufficient Representations for High‐Throughput Measurement: Lessons from Lossless Compression Benchmarks in 4D‐STEM. Advanced Computing. 2026;1(1). doi:10.1002/adco.70015
BibTeX
@article{Dyck2026,
author = {Dyck, Ondrej and Lupini, Andrew R. and Borisevich, Albina and Chi, Miaofang and Vasudevan, Rama K. and Jesse, Stephen},
title = {Inference‐Sufficient Representations for High‐Throughput Measurement: Lessons from Lossless Compression Benchmarks in 4D‐STEM},
journal = {Advanced Computing},
year = {2026},
volume = {1},
number = {1},
doi = {10.1002/adco.70015}
}
EUVIMED