夜间红外与可见光多尺度信息注入式图像融合

打开文本图片集

doi：10.37188/0PE.20253302.0282 CSTR：32169.14.OPE.20253302.0282

Multiscale semantic injective fusion of nighttime infrared and visible

YANG Yanchun*，LI Jialong，LI Yi，WANG Zeyu （School of Electronical and Information Engineering，Lanzhou Jiaotong University， Lanzhou 730070，China） *Correspondingauthor，E-mail：yangyanchunlO2@sina.com

Abstract： Aiming at the problems of unclear texture details and poor visual perception due to neglecting illumination in infrared and visible image fusion under low-light conditions，a low-light enhancement and semantic injection multi-scale infrared and visible image fusion method was proposed.Firstly，a network suitable for low-light enhancement was designed to realize the enhancement of visible images in nighttime scenes through repeated iterations of residual models.Then，a feature extractor based onthe Nest architecture was used as the encoder and decoder of the network，in which the deep features could capture the complex structure and semantic information of the images. A semantic prior learning module was designed to further extract the semantic information of the deep infrared and visible images through cross-attention， and a semantic injection unit was adopted to inject the enhancement features into each scale step by step.

Thirdly，a gradient enhancement branch was designed，where the mainstream features were first passed through hybrid atention，and then the Sobel operator stream and Laplacian operator stream were divided from the mainstream as a way to enhance the gradient of the fused image.Finally，the features at each scale were reconstructed by dense connections between the same layers and jump connections between different layers in thedecoder.Experimental results show that this method improves the visual information fidelity，mutual information，disparity correlation coeficient，and spatial frequency，on average，by （204号 23.1% ， 16.3% ， 18% ，and 39.8% ，respectively，in comparison with the nine methods，which effectively enhances the quality offused images in low-light environments，and helps to improve the performance of the advanced visual tasks.

Key words： infrared and visible image fusion；multiscale fusion networks；low-light enhancement；crossattention；semantic injection

1引言

由于技术限制和拍摄环境的影响，同一设备拍摄的单幅图像往往无法全面描述整个场景。（剩余15845字）