EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

Inference‐Sufficient Representations for High‐Throughput Measurement: Lessons from Lossless Compression Benchmarks in 4D‐STEM

Ondrej Dyck, Andrew R. Lupini, Albina Borisevich, Miaofang Chi, Rama K. Vasudevan, Stephen Jesse

Advanced Computing · 2026-12 · 1 (1)

DOI: 10.1002/adco.70015 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Advanced Computing, online erschienen am 05.11.2026. Verfasst von Ondrej Dyck und 5 weiteren Autor:innen. Die Arbeit nennt 36 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: Basic Energy Sciences, Laboratory Directed Research and Development.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Four‐dimensional scanning transmission electron microscopy (4D‐STEM) generates datasets whose volume already outpaces practical storage, transfer, and analysis. We benchmark 13 off‐the‐shelf lossless compression implementations across five datasets and find that the Blosc family substantially outperforms the compressors built into HDF5, achieving comparable compression ratios at far higher throughput. The broader conclusion is that this route is closing. Lossless compression is bounded by the entropy of the source, and on a simulated dataset where that bound can be computed exactly, the best implementation already comes within a factor of 2.5. A better implementation would recover a small multiple, not the orders of magnitude that rising detector rates demand. What must be decided instead is which data to keep and which to discard. Researchers already make this judgement routinely and often intuitively; it will have to be made explicit as agentic systems take on data collection and analysis. We argue for inference‐sufficient representations: a record suffices when it retains what an intended inference requires, not when it is complete. We distinguish coarsening, which cannot be false, from interpretation, which can, and show that event‐based detectors already perform the latter. We suggest design principles for deciding what a given inference requires.

Förderung

  • Basic Energy Sciences
  • Laboratory Directed Research and Development

Zitieren

APA

Dyck, O., Lupini, A. R., Borisevich, A., Chi, M., Vasudevan, R. K., & Jesse, S. (2026). Inference‐Sufficient Representations for High‐Throughput Measurement: Lessons from Lossless Compression Benchmarks in 4D‐STEM. Advanced Computing, 1(1). https://doi.org/10.1002/adco.70015

Vancouver

Dyck O, Lupini AR, Borisevich A, Chi M, Vasudevan RK, Jesse S. Inference‐Sufficient Representations for High‐Throughput Measurement: Lessons from Lossless Compression Benchmarks in 4D‐STEM. Advanced Computing. 2026;1(1). doi:10.1002/adco.70015

BibTeX
@article{Dyck2026,
  author = {Dyck, Ondrej and Lupini, Andrew R. and Borisevich, Albina and Chi, Miaofang and Vasudevan, Rama K. and Jesse, Stephen},
  title = {Inference‐Sufficient Representations for High‐Throughput Measurement: Lessons from Lossless Compression Benchmarks in 4D‐STEM},
  journal = {Advanced Computing},
  year = {2026},
  volume = {1},
  number = {1},
  doi = {10.1002/adco.70015}
}
Kostenlos starten →