A Multi-Stage Feature Enhancement RGB-T Object Tracking Method Based on Transformer

Visible Light-Thermal Infrared (RGB-T) target tracking aims to combine the dual advantages of visible light and thermal infrared imaging for stable tracking performance. Common methods assign weights to each modality, but this fails to fully utilize their complementarity. We propose a multi-stage multi-modal fusion method for RGB-T target tracking. First, two independent feature extraction networks are used to extract respective modal features, extending the single-modal tracker TransT to the RGB-T task. Second, we design intra-modal multi-scale feature aggregation and inter-modal feature modulation fusion. The former exploits the feature complementarity of multi-scale information within a single modality, enhancing the discriminative ability of single-modal features. The latter aims to utilize the feature complementarity of multi-modal information for interactive multi-modal feature integration. Our method achieves a success rate of 57.3% and an accuracy of 71.2% on the GTOT dataset, and a success rate of 44.6% and an accuracy of 56.8% on the LasHeR dataset. Experimental results indicate that the proposed tracking algorithm effectively improves target tracking performance.