| Citation: |
DI Jing, LIANG Chan, REN Li, GUO Wenqing, LIAN Jing. Infrared and Visible Image Fusion Based on Multi-Scale Contrast Enhancement and Cross-Dimensional Interactive Attention Mechanism[J]. Infrared Technology , 2024, 46(7): 754-764.
|
Addressing the issues of inadequate feature extraction, lack of saliency in fused image regions, and missing detailed information in infrared-visible image fusion, this paper proposes a method for infrared-visible image fusion based on multi-scale contrast enhancement and a cross-modal interactive attention mechanism. The main components of the proposed method are as follows. 1) Multi-scale contrast enhancement module: Designed to strengthen the intensity information of target regions, facilitating the fusion of complementary information from both infrared and visible images. 2) Dense connection block: Employed for feature extraction to minimize information loss and maximize information utilization. 3) Cross-modal interactive attention mechanism: Developed to capture crucial information from both modalities and enhance the performance of the network. 4) Decomposition network: Designed to decompose the fused image back into source images, incorporating more scene details and richer texture information into the fused image. The proposed fusion framework was experimentally evaluated on the TNO dataset. The results show that the fused images obtained by this method feature significant target regions, rich detailed textures, better fusion performance, and stronger generalization ability. Additionally, the proposed method outperforms other compared algorithms in both subjective performance and objective evaluation.
| [1] |
唐霖峰, 张浩, 徐涵, 等. 基于深度学习的图像融合方法综述[J]. 中国图象图形学报, 2023, 28(1): 3-36. https://www.cnki.com.cn/Article/CJFDTOTAL-ZGTB202301002.htm
TANG Linfeng, ZHANG Hao, XU Han, et al. A review of image fusion methods based on deep learning[J]. Journal of Image and Graphics, 2023, 28(1): 3-36. https://www.cnki.com.cn/Article/CJFDTOTAL-ZGTB202301002.htm
|
| [2] |
SUN Haijiang, LIU Qiaoyuan, WANG Jiacheng, et al. Fusion of infrared and visible images for remote detection of low-altitude slow-speed small targets[J]. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 2021, 14: 2971-2983.
|
| [3] |
SHUAI Xincheng, JING Zhongliang, TUO Hongya. SAFuseNet: integration of fusion and detection for infrared and visible images[J]. Aerospace Systems, 2022, 5(4): 655-661.
|
| [4] |
LI H, WU X J. CrossFuse: A novel cross attention mechanism based infrared and visible image fusion approach[J]. Information Fusion, 2024, 103: 102147.
|
| [5] |
ZHOU Tao, LI Qi, LU Huiling, et al. GAN review: models and medical image fusion applications[J]. Information Fusion, 2023, 91: 134-148. DOI: 10.1016/j.inffus.2022.10.017
|
| [6] |
HU Ke, SUN Wenhao, NIE Zhongbo, et al. Real-time infrared small target detection network and accelerator design[J]. Integration, 2022, 87: 241-252. DOI: 10.1016/j.vlsi.2022.07.008
|
| [7] |
孙彬, 诸葛吴为, 高云翔, 等. 基于潜在低秩表示的红外和可见光图像融合[J]. 红外技术, 2022, 44(8): 853-862. http://hwjs.nvir.cn/cn/article/id/7fc3a60d-61bb-454f-ad00-e925eeb54576
SUN Bin, ZHUGE Wuwei, GAO Yunxiang, et al. Infrared and visible image fusion based on potential low-rank representation[J]. Infrared Technology, 2022, 44(8): 853-862. http://hwjs.nvir.cn/cn/article/id/7fc3a60d-61bb-454f-ad00-e925eeb54576
|
| [8] |
TAN W, ZHOU H, SONG J, et al. Infrared and visible image perceptive fusion through multi-level Gaussian curvature filtering image decomposition[J]. Applied Optics, 2019, 58(12): 3064-3073.
|
| [9] |
Archana R, Jeevaraj P S E. Deep learning models for digital image processing: a review[J]. Artificial Intelligence Review, 2024, 57(1): 11. DOI: 10.1007/s10462-023-10631-z
|
| [10] |
WANG Chang, WU Yang, YU Yi, et al. Joint patch clustering-based adaptive dictionary and sparse representation for multi-modality image fusion[J]. Machine Vision and Applications, 2022, 33(5): 69. DOI: 10.1007/s00138-022-01322-w
|
| [11] |
CHEN Yudan, WANG Yuanbo, HUANG Fuyu, et al. Infrared and Visible Images Fusion base on Wavelet Transform[J]. Sixth Symposium on Novel Optoelectronic Detection Technology and Applications, 2020, 11455: 875-882.
|
| [12] |
LI Hui, WU Xiaojun, Kittler Josef. MDLatLRR: A novel decomposition method for infrared and visible image fusion. [J]. IEEE Transactions on Image Processing, 2020, 29: 4733-4746. DOI: 10.1109/TIP.2020.2975984
|
| [13] |
LIU Yu, CHEN Xun, CHENG Juan, et al. Infrared and visible image fusion with convolutional neural networks[J]. International Journal of Wavelets, Multiresolution and Information Processing, 2018, 16(3): 1850018.
|
| [14] |
ZHAO Z, XU S, ZHANG J, et al. Efficient and model-based infrared and visible image fusion via algorithm unrolling[J]. IEEE Transactions on Circuits and Systems for Video Technology, 2021, 32(3): 1186-1196.
|
| [15] |
LI Hui, WU Xiaojun, Kittler Josef. RFN-Nest: An end-to-end residual fusion network for infrared and visible images[J]. Information Fusion, 2021, 73: 72-86. DOI: 10.1016/j.inffus.2021.02.023
|
| [16] |
TANG Linfeng, YUAN Jiteng, ZHANG Hao, et al. PIAFusion: a progressive infrared and visible image fusion network based on illumination aware[J]. Information Fusion, 2022, 83: 79-92.
|
| [17] |
TANG Linfeng, XIANG Xinyu, ZHANG Hao, et al. DIVFusion: Darkness-free infrared and visible image fusion[J]. Information Fusion, 2023, 91: 477-493. http://www.nstl.gov.cn/paper_detail.html?id=c2c499247ee68b7a6ee8622c95b62724
|
| [18] |
Kayadibi İsmail, Güraksın Gür Emre. An explainable fully dense fusion neural network with deep support vector machine for retinal disease determination[J]. International Journal of Computational Intelligence Systems, 2023, 16(1): 28. DOI: 10.1007/s44196-023-00210-z
|
| [19] |
LI H, WU X J, DURRANI T S. Infrared and visible image fusion with ResNet and zero-phase component analysis[J]. Infrared Physics & Technology, 2019, 102: 103039.
|
| [20] |
LI Hui, WU Xiaojun, Tariq Durrani. NestFuse: an infrared and visible image fusion architecture based on nest connection and spatial/channel attention models[J]. IEEE Transactions on Instrumentation and Measurement, 2020, 69(12): 9645-9656.
|
| [21] |
ZHAO Zixiang, XU Shuang, ZHANG Chunxia, et al. Bayesian fusion for infrared and visible images[J]. Signal Processing, 2020, 177: 107734. DOI: 10.1016/j.sigpro.2020.107734
|
| [22] |
XUE Weimin, WANG Anhong, ZHAO Lijun. FLFuse-Net: A fast and lightweight infrared and visible image fusion network via feature flow and edge compensation for salient information[J]. Infrared Physics and Technology, 2022, 127: 104383.
|
| [23] |
Veshki Farshad G, Ouzir Nora, Vorobyov Sergiy A, et al. Multimodal image fusion via coupled feature learning[J]. Signal Processing, 2022, 200: 108637.
|
| [1] | LIU Lei, QIAN Yunsheng. A Low Illumination Image Acquisition and Processing System Based on FPGA[J]. Infrared Technology , 2022, 44(5): 462-468. |
| [2] | SUN Shaowei, YANG Yuetao, YANG Bingwei, WAN Anjun, ZHONG Hailin. Research and Implementation of Infrared Lens Auto-focus Technology Based on Field Programmable Gate Array[J]. Infrared Technology , 2021, 43(5): 464-472. |
| [3] | CHI Linhui, QIAN Yunsheng, JI Yuhao. Verification Protocol for Improving Communication Stability Between FPGAs[J]. Infrared Technology , 2020, 42(11): 1022-1027. |
| [4] | High-speed Spectrum Inversion System Based on FPGA[J]. Infrared Technology , 2019, 41(6): 535-539. |
| [5] | LIU Yuan, LI Qing, LIANG Yanju. Implementation of Infrared Target Detection System Based on FPGA[J]. Infrared Technology , 2019, 41(6): 521-526. |
| [6] | LIU Jiangping, XUE Heru. High-speed Spectrum Acquisition and Processing System Based on FPGA[J]. Infrared Technology , 2018, 40(11): 1042-1046. |
| [7] | ZHANG Chenghong, LI Fanming, YANG Long. Real Time Infrared Video Capture and Display System Based on FPGA[J]. Infrared Technology , 2017, 39(2): 143-146. |
| [8] | LIU Rui-qiang, WANG Yong-xin. Based on FPGA Real-time Spectrum Obtained of Static Fourier Spectrometer[J]. Infrared Technology , 2011, 33(8): 465-469. DOI: 10.3969/j.issn.1001-8891.2011.08.008 |
| [9] | GONG Man-man, CHEN Qian, GU Guo-hua, SUI Xiu-bao. FPGA-Based Realization of Second-Order Newton Interpolation of Infrared Image[J]. Infrared Technology , 2010, 32(12): 723-726. DOI: 10.3969/j.issn.1001-8891.2010.12.009 |
| [10] | Improved Canny Edge Detection Algorithm and Implementation in FPGA[J]. Infrared Technology , 2010, 32(2): 93-96. DOI: 10.3969/j.issn.1001-8891.2010.02.008 |
| 1. |
朱强，周维虎，陈晓梅，石俊凯，李冠楠. 高速实时近红外弱信号检测系统. 光学精密工程. 2022(24): 3116-3127 .
|
Supported by: Beijing Renhe Information Technology Co., Ltd. 
DownLoad: