基于策略梯度和伪孪生网络的异源图像匹配

打开文本图片集

中图分类号：TP394. 1；TH691. 9 文献标识码：Adoi：10. 37188/OPE. 20253306. 0945 CSTR：32169. 14. OPE. 20253306. 0945

Cross-modality image matching algorithm based on policy gradient and pseudo-twin network

ZHANG Jian 1，2，3 ，LIANG Ao 1，2，3 ，HUA Haiyang 1，2* ，LIU Tianc 1，2 ，LI Shihan 1，2，3

（1. Key Laboratory of Opto-Electronic Information Processing，Chinese Academy of Sciences， Shenyang 110016，China；

2. Shenyang Institute of Automation，Chinese Academy of Sciences，Shenyang 110016，China；

3. University of Chinese Academy of Sciences，Beijing ，China） * Corresponding author，E-mail：c3ill@sia. cn

Abstract：A multi-source image matching method named PCMM-Net was proposed to address the prob⁃ lem of unmatched keypoints resulting from the different imaging mechanisms of visible and infrared imag⁃ es. Firstly，a U-Net model with a policy gradient mechanism was introduced as the baseline model to ex⁃ tract keypoints from the images. This foundational model transformed pixel values into normalized proba⁃ bilities，serving to filter out low-texture areas. This process enabled the network to focus on and learn key⁃ points that were both reliable and repeatable. Then，to address the radiance discrepancies between visible images and infrared images，a pseudo-twin network was employed to extract similar features from local im age patches. Finally，a fusion layer was proposed to integrate similar features and features from keypoint detectors，generating descriptors suitable for multi-source image matching. The proposed algorithm was validated for matching performance on the VEDAI near-infrared dataset and the MTV thermal infrared da⁃ taset. Experimental results demonstrate that the proposed algorithm achieves average matching accuracies of 97.77% and 95.88% on the VEDAI and MTV datasets，respectively. Compared to the DALF algo⁃ rithm，the average matching accuracies are improved by 2.26% and 14% on VEDAI and MTV datasets. Experimental results show that the algorithm has better matching effect and improves the accuracy of matching.

Key words：visible and infrared images； multi-source image matching； deep learning； ConvolutionalNeural Network（CNN）

1 引 言

异源图像匹配算法主要可以分为可见光红外图像匹配、可见光和SAR 图像匹配等，本文主要针对可见光与红外图像。（剩余21820字）