EUVIMEDEUVIMED entdecken
Crossref

Visible–Infrared Dual-Sensor Fusion for Single-Object Tracking

Weichun Liu, Weibing Liu, Yuxin Sun

IEEE Sensors Journal · 15.02.2023 · 23 (4) · 4118-4128

DOI: 10.1109/jsen.2023.3234091 ↗

Kurz gefasst

Fachartikel in der Zeitschrift IEEE Sensors Journal, erschienen am 15.02.2023. Verfasst von Weichun Liu und 2 weiteren Autor:innen. Die Arbeit wurde bisher 17-mal zitiert (Crossref) und nennt 39 Literaturangaben. Gefördert von: Natural Science Foundation of Hunan Province of China.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Visual object tracking is a classical yet rapidly evolving task in computer vision. Traditional RGB trackers in the visible spectrum suffer from poor lighting conditions and bad weather, which significantly affect image quality. To tackle this drawback, RGB trackers are extended from single modality (RGB) to dual modalities [RGB-thermal (RGB-T)] via visible–infrared dual-sensor fusion. Compared with visible cameras, thermal infrared cameras are insensitive to lightning conditions and have a strong ability to penetrate haze and smog. Inspired by the complementary merits of visible and thermal information, we propose a visible–infrared (RGB-T) tracker that fuses the visible modality and infrared modality on the pixel level, feature level, and decision level. First, for the pixel-level fusion, a fused histogram is derived from a four-channel RGB-T image that concatenates an aligned RGB and thermal image pair. With the fused histogram, a foreground mask is derived for guiding correlation filter learning by segmenting the foreground target from the neighbor background. Second, for the feature-level fusion, handcrafted features, such as histogram of gradient (HOG) and color name (CN), are extracted from the RGB and thermal images, respectively, and then concatenated along the channel dimension for feature fusion. Third, for the decision-level fusion, the channel weight of each feature channel in both modalities is estimated for weighting per-channel filter response in localization. To demonstrate the effectiveness of our visible and infrared fusion strategy, we perform an ablation study to evaluate each level of fusion independently on the RGBT234 dataset. We also perform extensive experiments on the RGBT234 and VTUAV_ST datasets to evaluate the performance of the proposed RGB-T tracker compared with state-of-the-art trackers. It is worth noting that our three-level visible–infrared fusion framework is generic and flexible and our implementation is far from optimal. This framework is open-source and easy to be combined with any correlation filter-based RGB-T tracker for further performance gain.

Förderung

  • Natural Science Foundation of Hunan Province of China

Zitieren

APA

Liu, W., Liu, W., & Sun, Y. (2023). Visible–Infrared Dual-Sensor Fusion for Single-Object Tracking. IEEE Sensors Journal, 23(4), 4118-4128. https://doi.org/10.1109/jsen.2023.3234091

Vancouver

Liu W, Liu W, Sun Y. Visible–Infrared Dual-Sensor Fusion for Single-Object Tracking. IEEE Sensors Journal. 2023;23(4):4118-4128. doi:10.1109/jsen.2023.3234091

BibTeX
@article{Liu2023,
  author = {Liu, Weichun and Liu, Weibing and Sun, Yuxin},
  title = {Visible–Infrared Dual-Sensor Fusion for Single-Object Tracking},
  journal = {IEEE Sensors Journal},
  year = {2023},
  volume = {23},
  number = {4},
  pages = {4118--4128},
  doi = {10.1109/jsen.2023.3234091}
}
Kostenlos starten →