EUVIMEDSkopri EUVIMED
Open access · CC BY 4.0

Absorbed or Anchored? How Large Language Models Represent Embedded Words in Modern Standard Arabic–Egyptian Arabic Code-Switching

Fahad AlGhamdi

Electronics · 2026-11-05 · 15 (21) · 5141

DOI: 10.3390/electronics15215141 ↗

At a glance

Journal article in Electronics, published online on 2026-11-05. Written by Fahad AlGhamdi. 78 references. The full text is freely available (open access, CC BY 4.0).

AstrattDettalji biblijografiċiCiteRelated studies

Abstract (original language)

We study how linear probes read Modern Standard Arabic (MSA)–Egyptian Arabic code-switching in Linguistic Code-switching Evaluation (LinCE) tweets using Qwen3-8B-Base, ALLaM-7B-Instruct-preview, Gemma-2-9B and Fanar-1-9B. Probes trained on monolingual text classify 54–56% of MSA words in Egyptian contexts as Egyptian, but only 4–8% of Egyptian words in MSA contexts as MSA. Lexical and contextual controls reduce the gap, leaving 12–20 percentage points after regression and 19–28 after matching. Probes trained on balanced code-switched data recover the inserted word’s variety with 96–98% accuracy, so the asymmetry does not imply loss of variety information. Probe readings correlate with next-token variety preference. Replacing the outputs of five patching-selected attention heads with reference means changes both measures relative to layer-matched random heads, but reduces MSA-to-Egyptian misclassification by only 1–14 percentage points. Effects change direction with a different replacement mean and also occur in monolingual text. External news comments reproduce the directional readout gap (38–69 points), although the difference between lexically matched directional shifts remains unresolved. Register remains a possible contributor, and head selection and interventions use one tweet corpus. Probe-based analyses should therefore report switching directions separately and distinguish monolingual readouts from recoverable word variety.

Cite

APA

AlGhamdi, F. (2026). Absorbed or Anchored? How Large Language Models Represent Embedded Words in Modern Standard Arabic–Egyptian Arabic Code-Switching. Electronics, 15(21), 5141. https://doi.org/10.3390/electronics15215141

Vancouver

AlGhamdi F. Absorbed or Anchored? How Large Language Models Represent Embedded Words in Modern Standard Arabic–Egyptian Arabic Code-Switching. Electronics. 2026;15(21):5141. doi:10.3390/electronics15215141

BibTeX
@article{AlGhamdi2026,
  author = {AlGhamdi, Fahad},
  title = {Absorbed or Anchored? How Large Language Models Represent Embedded Words in Modern Standard Arabic–Egyptian Arabic Code-Switching},
  journal = {Electronics},
  year = {2026},
  volume = {15},
  number = {21},
  pages = {5141},
  doi = {10.3390/electronics15215141}
}
Start for free →