EUVIMEDDescobrir EUVIMED
Open access · CC BY 4.0

HEADHUNTER: Training-Free Annotated Dataset Synthesis via Self-Guided Diffusion Transformer Attention Head Selection

Rohan Le Roux, Siavash Khaksar, Mohammadali Sepehri, Iain Murray

Machine Learning and Knowledge Extraction · 2026-08-07 · 8 (8) · 230

DOI: 10.3390/make8080230 ↗

At a glance

Journal article in Machine Learning and Knowledge Extraction, published online on 2026-08-07. Written by Rohan Le Roux and 3 co-authors. 44 references. The full text is freely available (open access, CC BY 4.0).

ResumoDados bibliográficosCiteRelated studies

Abstract (original language)

Pixel-level annotation remains a major bottleneck for semantic segmentation, motivating methods that synthesize image–label pairs directly from generative models. Prior synthetic dataset generators typically obtain pseudo-labels from cross-attention maps or learned decoders over generative features; however, recent text-to-image (T2I) models increasingly use multimodal diffusion transformers (MM-DiTs), where concept localization is no longer exposed through a single cross-attention pathway but instead distributed across many layers and attention heads. Existing MM-DiT localization methods address this by aggregating saliency across heads, but we observe that this averaging can dilute clean target localizers due to attention head heterogeneity. We introduce HEADHUNTER, a training-free segmentation framework that uses aggregate concept saliency as a self-guided proxy to select the single attention head that best localizes a queried textual concept, yielding cleaner segmentation masks. We then use HEADHUNTER to turn target classes into training data automatically: a large language model (LLM) diversifies prompts, an MM-DiT generates images, HEADHUNTER produces pseudo-labels, and a vision language model (VLM) verifies each image–mask pair before acceptance. HEADHUNTER achieves strong zero-shot segmentation performance (81.2 mIoU on PASCAL VOC2012 and 73.2 mIoU on ImageNet-Segmentation), outperforming head aggregation and other interpretability methods. Using only our generated image–label pairs, we train segmentation models which reach 64.9 mIoU on VOC2012 validation, matching or outperforming comparable synthetic dataset generators and showing that the proposed pipeline produces effective dense labels without human intervention.

Cite

APA

Roux, R. L., Khaksar, S., Sepehri, M., & Murray, I. (2026). HEADHUNTER: Training-Free Annotated Dataset Synthesis via Self-Guided Diffusion Transformer Attention Head Selection. Machine Learning and Knowledge Extraction, 8(8), 230. https://doi.org/10.3390/make8080230

Vancouver

Roux RL, Khaksar S, Sepehri M, Murray I. HEADHUNTER: Training-Free Annotated Dataset Synthesis via Self-Guided Diffusion Transformer Attention Head Selection. Machine Learning and Knowledge Extraction. 2026;8(8):230. doi:10.3390/make8080230

BibTeX
@article{Roux2026,
  author = {Roux, Rohan Le and Khaksar, Siavash and Sepehri, Mohammadali and Murray, Iain},
  title = {HEADHUNTER: Training-Free Annotated Dataset Synthesis via Self-Guided Diffusion Transformer Attention Head Selection},
  journal = {Machine Learning and Knowledge Extraction},
  year = {2026},
  volume = {8},
  number = {8},
  pages = {230},
  doi = {10.3390/make8080230}
}
Start for free →