EUVIMEDEUVIMED entdecken
Crossref

Accuracy Is Not Capability: A Causal Audit Standard for RF Classification

Atul Rawal, Adrienne Raglin

Proceedings of the AAAI Symposium Series · 06.11.2026 · 10 (1) · 402-410

DOI: 10.1609/aaaiss.v10i1.43432 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Proceedings of the AAAI Symposium Series, online erschienen am 06.11.2026. Verfasst von Atul Rawal und 1 weiteren Autor:innen.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

A radio-frequency (RF) classifier that performs well on held-out test data has not necessarily learned meaningful characteristics of the emitter itself. Instead, it may be relying on patterns created by how the data was collected. In this study, we present a four-stage audit designed to tell these two cases apart: checking for shortcuts, accounting for confounding variables, examining how variables depend on one another, and testing the model under controlled changes. We apply this audit to a nine-class automatic modulation classification (AMC) dataset containing hardware-emulated raw IQ data, where the audit is able to establish several specific and testable findings. Higher-order cumulant features remain stable under changes in gain and phase, the discovered dependency graph connects the class label to modulation-related statistics rather than channel variables, and the estimated effect of signal-to-noise ratio (SNR) under randomized intervention is about ten times larger than the estimate obtained by simply observing the original data. One of the important findings is a specific failure mode in which a commonly used SNR estimator, which assumes a constant-modulus signal, can unintentionally act like a modulation classifier. As a result, information about the true class can leak into a variable that appears to measure only the channel conditions. Applying the same audit to a UAV dataset containing only preprocessed spectral data and no collection metadata shows the limitation at the other end of the audit ladder. In this case, the audit can determine that the reported classification accuracy is strongly influenced by differences in signal energy, but it cannot determine how well the classifier would perform if that shortcut were removed. The difference between these two outcomes comes from what information the datasets provide, rather than from the audit method itself. We organize these capabilities into a three-level audit ladder and use it to provide practical recommendations for how future RF datasets should be collected.

Zitieren

APA

Rawal, A., & Raglin, A. (2026). Accuracy Is Not Capability: A Causal Audit Standard for RF Classification. Proceedings of the AAAI Symposium Series, 10(1), 402-410. https://doi.org/10.1609/aaaiss.v10i1.43432

Vancouver

Rawal A, Raglin A. Accuracy Is Not Capability: A Causal Audit Standard for RF Classification. Proceedings of the AAAI Symposium Series. 2026;10(1):402-410. doi:10.1609/aaaiss.v10i1.43432

BibTeX
@article{Rawal2026,
  author = {Rawal, Atul and Raglin, Adrienne},
  title = {Accuracy Is Not Capability: A Causal Audit Standard for RF Classification},
  journal = {Proceedings of the AAAI Symposium Series},
  year = {2026},
  volume = {10},
  number = {1},
  pages = {402--410},
  doi = {10.1609/aaaiss.v10i1.43432}
}
Kostenlos starten →