EUVIMEDEUVIMED entdecken
Crossref

Multi-Task Learning With Multi-Query Transformer for Dense Prediction

Yangyang Xu, Xiangtai Li, Haobo Yuan, Yibo Yang, Lefei Zhang

IEEE Transactions on Circuits and Systems for Video Technology · 2024-02 · 34 (2) · 1228-1240

DOI: 10.1109/tcsvt.2023.3292995 ↗

Kurz gefasst

Fachartikel in der Zeitschrift IEEE Transactions on Circuits and Systems for Video Technology, 2024-02. Verfasst von Yangyang Xu und 4 weiteren Autor:innen. Die Arbeit wurde bisher 74-mal zitiert (Crossref) und nennt 72 Literaturangaben. Gefördert von: National Natural Science Foundation of China, Special Fund of the Hubei Luojia Laboratory.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Previous multi-task dense prediction studies developed complex pipelines such as multi-modal distillations in multiple stages or searching for task relational contexts for each task. The core insight beyond these methods is to maximize the mutual effects of each task. Inspired by the recent query-based Transformers, we propose a simple pipeline named Multi-Query Transformer (MQTransformer) that is equipped with multiple queries from different tasks to facilitate the reasoning among multiple tasks and simplify the cross-task interaction pipeline. Instead of modeling the dense per-pixel context among different tasks, we seek a task-specific proxy to perform cross-task reasoning via multiple queries where each query encodes the task-related context. The MQTransformer is composed of three key components: shared encoder, cross-task query attention module and shared decoder. We first model each task with a task-relevant query. Then both the task-specific feature output by the feature extractor and the task-relevant query are fed into the shared encoder, thus encoding the task-relevant query from the task-specific feature. Secondly, we design a cross-task query attention module to reason the dependencies among multiple task-relevant queries; this enables the module to only focus on the query-level interaction. Finally, we use a shared decoder to gradually refine the image features with the reasoned query features from different tasks. Extensive experiment results on two dense prediction datasets (NYUD-v2 and PASCAL-Context) show that the proposed method is an effective approach and achieves state-of-the-art results. Code and models are available at https://github.com/yangyangxu0/MQTransformer.

Förderung

  • National Natural Science Foundation of China
  • Special Fund of the Hubei Luojia Laboratory

Zitieren

APA

Xu, Y., Li, X., Yuan, H., Yang, Y., & Zhang, L. (2024). Multi-Task Learning With Multi-Query Transformer for Dense Prediction. IEEE Transactions on Circuits and Systems for Video Technology, 34(2), 1228-1240. https://doi.org/10.1109/tcsvt.2023.3292995

Vancouver

Xu Y, Li X, Yuan H, Yang Y, Zhang L. Multi-Task Learning With Multi-Query Transformer for Dense Prediction. IEEE Transactions on Circuits and Systems for Video Technology. 2024;34(2):1228-1240. doi:10.1109/tcsvt.2023.3292995

BibTeX
@article{Xu2024,
  author = {Xu, Yangyang and Li, Xiangtai and Yuan, Haobo and Yang, Yibo and Zhang, Lefei},
  title = {Multi-Task Learning With Multi-Query Transformer for Dense Prediction},
  journal = {IEEE Transactions on Circuits and Systems for Video Technology},
  year = {2024},
  volume = {34},
  number = {2},
  pages = {1228--1240},
  doi = {10.1109/tcsvt.2023.3292995}
}
Kostenlos starten →