EUVIMEDScopri EUVIMED
Open access · CC BY 4.0

FLAMA: Frame-Level Alignment Margin Attack for Scene Text and Automatic Speech Recognition

Yikun Xu, Zhiheng Xu, Pengwen Dai

Electronics · 2026-03-04 · 15 (5) · 1064

DOI: 10.3390/electronics15051064 ↗

At a glance

Journal article in Electronics, published online on 2026-03-04. Written by Yikun Xu and 2 co-authors. Cited 1 times so far (Crossref); 56 references. The full text is freely available (open access, CC BY 4.0). Funded by: Youth Fund of the National Natural Science Foundation of China, Fundamental Research Funds for the Central Universities, Foundation of Yunnan Key Laboratory of Smart Education.

AbstractDati bibliograficiCiteRelated studies

Abstract (original language)

Scene text recognition (STR) and automatic speech recognition (ASR) translate visual or acoustic signals into linguistic sequences and underpin many modern perception systems. Although their front-ends and decoders differ (e.g., CTC-based, attention-based, or variants), both tasks ultimately rely on aligning input frames to output tokens by deep learning techniques, which exposes a shared vulnerability to adversarial perturbations. Existing attacks commonly optimize global sequence-level objectives. As a result, decisive frames are treated implicitly, and optimization can become unnecessarily diffuse over long input sequences, hindering convergence and perceptual quality. To address the above issues, we propose FLAMA, a unified Frame-Level Alignment Margin Attack, which could be used for both STR and ASR models. FLAMA explicitly targets alignment by maximizing per frame (or per step) recognition margins. The design is decoder-agnostic and applies to both CTC-based and attention-based pipelines. It employs a recognition-score-aware Step/Halt gate that concentrates updates on the most critical frames, and a stabilization stage that suppresses late-iteration oscillations to improve optimization stability and perceptual control. Ablation analyses show that stabilization consistently enhances attack success and reduces distortion. We evaluate FLAMA on STR benchmarks (SVT, CUTE80, and IC13) with CRNN, STAR, and TRBA, and on the ASR benchmark (LibriSpeech) with a Wav2Vec 2.0 model. Across modalities and architectures, FLAMA achieves near-100% attack success while substantially reducing l2 distortion and improving perceptual metrics compared with FGSM/PGD baselines. These results highlight frame-level alignment as a shared weak point across visual and audio sequence recognizers and suggest localized margin objectives as a principled route to effective sequence attacks.

Funding

  • Youth Fund of the National Natural Science Foundation of China
  • Fundamental Research Funds for the Central Universities
  • Foundation of Yunnan Key Laboratory of Smart Education

Cite

APA

Xu, Y., Xu, Z., & Dai, P. (2026). FLAMA: Frame-Level Alignment Margin Attack for Scene Text and Automatic Speech Recognition. Electronics, 15(5), 1064. https://doi.org/10.3390/electronics15051064

Vancouver

Xu Y, Xu Z, Dai P. FLAMA: Frame-Level Alignment Margin Attack for Scene Text and Automatic Speech Recognition. Electronics. 2026;15(5):1064. doi:10.3390/electronics15051064

BibTeX
@article{Xu2026,
  author = {Xu, Yikun and Xu, Zhiheng and Dai, Pengwen},
  title = {FLAMA: Frame-Level Alignment Margin Attack for Scene Text and Automatic Speech Recognition},
  journal = {Electronics},
  year = {2026},
  volume = {15},
  number = {5},
  pages = {1064},
  doi = {10.3390/electronics15051064}
}
Start for free →