针对RGBT跟踪的特殊属性的跨模态 交互融合网络

打开文本图片集

Specialattribute-basedcross-modal interactive fusion network for RGBT tracking

SHAO Xiaoqiang，LI Hao*，LU Zhiyue，MA Bo，LIU Mingqian，HAN Zehui （College of Electrical and Control Engineering， Xi' an University of Science and Technology，Xi'an 710054，China） * Corresponding author， E -mail： 2670815399@qq.com

Abstract： RGBT target tracking has gained widespread application in fields such as video surveillnce and autonomous driving due to its robustness and resistance to ilumination and occlusion. By leveraging the common challenging atributes in infrared and visible light images and fully interacting between the two modalities，an efective tracking network was constructed，capable of overcoming the impacts of various adverse scenarios encountered during the tracking process. This network was composed of three modules： the specific attribute fusion module，the common atribute fusion module，and the cross-modality interaction module.The specific attribute fusion module enabled the network to extract modality-specific challenging attributes，effctively utilizing the advantages ofdifferent modalities.The common attribute fusion module extracted features that Were matched in both modalities during target tracking and adaptively aggregated this information. It assigned corresponding weights to each common challenging attribute，thereby enhancing the tracker's adaptability. The cross-modality interaction module incorporated common modality information into the specific modality information of infrared and visible light images，thus improving the network's robustness.To address the issue of information loss across diffrent modalities，the traditional ross-entropy loss was optimized to enhance focus on each modality and accelerate network convergence. The proposed network is tested on the GTOT，RGBT234，and LasHeR datasets，achieving an accuracy of 84.1% and a precision of 57.3% on the RGBT234 dataset， 52.3% and a precision of 39.1% on the Lasher dataset. The results demonstrate that the tracker has achieved commendable performance， which validates the effectiveness of the proposed method.

Key Words： object tracking； thermal infrared tracking；challenge attributes；modal interaction

1引言

目标跟踪是根据目标第一帧的位置进行初始定位来预测后续帧目标的位置，在智能机器人、自动驾驶1等领域中发挥了极大的作用。（剩余18301字）