Image Fusion Algorithm Based on Thermal Radiation Information Retention
-
摘要: 针对现有的红外与可见光图像融合算法无法很好地保留红外图像热辐射信息这一问题，提出了一种基于热辐射信息保留的图像融合算法。通过NSCT（non-subsampled contourlet transform）变换对红外与可见光图像进行多尺度分解，得到各自的高频子带和低频子带，可见光低频子带部分经拉普拉斯算子提取特征后与红外低频子带部分叠加得到融合图像的低频系数，高频部分使用基于点锐度和细节增强的融合规则进行融合以得到高频系数，最后通过逆NSCT变换重构得到融合图像。实验表明，相较于其它图像融合算法，所提算法能在保留红外图像热辐射信息的同时，保有较好的清晰细节表现能力，并在多项客观评价指标上优于其它算法，具有更好的视觉效果，且在伪彩色变换后有良好的视觉体验，验证了所提算法的有效性和可行性。Abstract: Focusing on the issue that existing algorithms of infrared and visible image fusion cannot retain thermal radiation information from infrared images, an image fusion algorithm based on thermal radiation information retention was proposed. Multi-scale decomposition of infrared and visible light images was performed through NSCT transformation to obtain the respective high-frequency sub-bands and low-frequency sub-bands. The low-frequency sub-bands of visible light were extracted by the Laplacian and superimposed with the infrared low-frequency sub-bands to obtain low-frequency sub-bands of the fused image. The fusion rule, which is based on point sharpness, and detail enhancement were used to obtain the high-frequency coefficients of the high-frequency part; the fused image was then reconstructed through inverse NSCT transformation. The experimental results indicate that compared with other image fusion algorithms, the proposed algorithm can retain the thermal radiation information of infrared images, while maintaining good performance with clear details, and is superior to other algorithms in several objective evaluation indices. The proposed algorithm has better visual effects and a good visual experience after pseudo-color transformation, which verifies the effectiveness and feasibility of the proposed algorithm.
-
Key words:
- infrared image /
- multi-scale decomposition /
- image fusion /
- pseudo-color transformation
-
Method AG EN SF SSIM MI VIF DWT 4.9666 6.6713 9.0816 0.7567 19.8462 0.5406 LP 5.2522 6.8009 9.8522 0.7454 16.4003 0.5501 NSCT 4.9564 7.3060 9.2742 0.6267 22.9055 0.5282 NSCT-PCNN 4.6766 7.2198 8.8717 0.6945 2.2669 0.7031 Proposed method 6.4685 7.0244 11.6793 0.8661 77.5039 0.7098 
下载: 导出CSV
Method AG EN SF SSIM MI VIF DWT 5.8641 6.4416 11.1577 0.5967 7.4241 0.2729 LP 6.2391 6.6517 12.0277 0.5715 7.7505 0.3038 NSCT 6.0105 7.2517 11.6053 0.4455 10.4365 0.2970 NSCT-PCNN 5.3421 7.3359 10.6143 0.5201 7.1469 0.4708 Proposed method 7.3643 6.6584 13.7557 0.6811 22.8122 0.3906
下载: 导出CSV
表 3 平均评价指标
Table 3. Average evaluation indexes
Method AG EN SF SSIM MI VIF DWT 6.1136 7.0445 11.9406 0.6936 9.1349 0.2918 LP 6.4499 7.3049 12.6930 0.6892 8.9399 0.3481 NSCT 6.1955 7.7048 12.1721 0.5986 10.3829 0.3176 NSCT-PCNN 5.5698 7.8429 11.3441 0.7352 8.5166 0.5246 Proposed method 7.6364 7.3516 14.6092 0.7520 24.9254 0.4028
下载: 导出CSV
-
[1] 谢春宇, 徐建, 李新德, 等. 基于深度学习的红外与可见光图像融合方法[J]. 指挥信息系统与技术, 2020, 11(2): 15-20, 38. https://www.cnki.com.cn/Article/CJFDTOTAL-ZHXT202002003.htmXIE Chunyu, XU Jian, LI Xinde, et al. Infrared and visible image fusion method based on deep learning[J]. Command Information System and Technology, 2020, 11(2): 15-20, 38. https://www.cnki.com.cn/Article/CJFDTOTAL-ZHXT202002003.htm [2] Rajiv S, Ashish K. Multiscale Medical image fusion in wavelet domain[J]. The Scientific World Journal, 2013, 2013: 521034. http://www.scienceopen.com/document_file/6e3eb747-a10e-42b1-8a95-0a0060e9b4b5/PubMedCentral/6e3eb747-a10e-42b1-8a95-0a0060e9b4b5.pdf [3] Padma Ganasala, Achanta Durga Prasad. Medical image fusion based on laws of texture energy measures in stationary wavelet transform domain[J]. International Journal of Imaging Systems and Technology, 2020, 30(3): 544-557. doi: 10.1002/ima.22393 [4] Do Minh N, Vetterli Martin. The contourlet transform: an efficient directional multiresolution image representation[J]. IEEE Transactions on Image Processing, 2005, 14(12): 1-16. doi: 10.1109/TIP.2005.861050 [5] Mertens T, Kaut Z J, Van Reeth F. Exposure fusion: a simple and practical alternative to high dynamic range photography[J]. Computer Graphics Forum, 2009, 28(1): 161-171. doi: 10.1111/j.1467-8659.2008.01171.x [6] Da Cunha A L, Zhou J, Do M N. The nonsubsampled contourlet transform: theory, design, and applications[J]. IEEE Transactions on Image Processing, 2006, 15: 3089-3101. doi: 10.1109/TIP.2006.877507 [7] XING Xiaoxue, LIU Cheng, LUO Cong, et al. Infrared and visible image fusion based on nonlinear enhancement and NSST decomposition[J]. EURASIP Journal on Wireless Communications and Networking, 2020, 2020(1): 694-700. doi: 10.1186/s13638-020-01774-6 [8] WANG Z, LI X, DUAN H, et al. Multifocus image fusion using convolutional neural networks in the discrete wavelet transform domain[J]. Multimedia Tools and Applications, 2019, 78(24): 34483-34512. doi: 10.1007/s11042-019-08070-6 [9] 杨艳春, 王艳, 党建武, 等. 基于RGF和改进自适应Unit-Linking PCNN的红外与可见光图像融合[J]. 光电子·激光, 2020, 31(4): 401-410. https://www.cnki.com.cn/Article/CJFDTOTAL-GDZJ202004010.htmYANG Yanchun, WANG Yan, DANG Jianwu, et al. Infrared and visible image fusion based on RGF and improved adaptive Unit-Linking PCNN[J]. Journal of Optoelectronics·Laser, 2020, 31(4): 401-410. https://www.cnki.com.cn/Article/CJFDTOTAL-GDZJ202004010.htm [10] 林玉池, 周欣, 宋乐, 等. 基于NSCT变换的红外与可见光图像融合技术研究[J]. 传感器与微系统, 2008, 27(12): 45-47. doi: 10.3969/j.issn.1000-9787.2008.12.015LIN Yuchi, ZHOU Xin, SONG Le, et al. Infrared and visible image fusion technology based on non subsampled contourlet transform[J]. Transducer and Microsystem Technologies, 2008, 27(12): 45-47. doi: 10.3969/j.issn.1000-9787.2008.12.015 [11] 程永翔, 刘坤, 贺钰博. 基于卷积神经网络与视觉显著性的图像融合[J]. 计算机应用与软件, 2020, 37(3): 225-230. doi: 10.3969/j.issn.1000-386x.2020.03.038CHENG Yongxiang, LIU Kun, HE Yubo. Image fusion with convolutional neural network and visual saliency[J]. Computer Applications and Software, 2020, 37(3): 225-230. doi: 10.3969/j.issn.1000-386x.2020.03.038 [12] 巩稼民, 刘爱萍, 马豆豆, 等. 结合邻域特征与IDCSCM的红外与可见光图像融合[J]. 激光与红外, 2020, 50(7): 889-896. https://www.cnki.com.cn/Article/CJFDTOTAL-JGHW202007020.htmGONG Jiamin, LIU Aiping, MA Doudou, et al. Infrared and visible image fusion combining neighborhood features with IDCSCM[J]. Laser & Infrared, 2020, 50(7): 889-896. https://www.cnki.com.cn/Article/CJFDTOTAL-JGHW202007020.htm [13] 孙英慧, 蒲东兵. 基于拉普拉斯算子的边缘检测研究[J]. 长春师范学院学报: 人文社会科学版, 2009, 28(12): 4-6. https://www.cnki.com.cn/Article/CJFDTOTAL-CCSS200912002.htmSUN Yinghui, PU Dongbing. Edge Detection Research on Laplace Operator[J]. Journal of Changchun Normal University: Humanities and Social Sciences Edition, 2009, 28(12): 4-6. https://www.cnki.com.cn/Article/CJFDTOTAL-CCSS200912002.htm [14] 彭土有, 吴洁, 彭俊. 拉普拉斯边缘检测算法的改进及其在探地雷达中的应用[J]. 现代雷达, 2020, 42(8): 41-45. https://www.cnki.com.cn/Article/CJFDTOTAL-XDLD202008008.htmPENG Tuyou, WU Jie, PENG Jun. Improvement of Laplacian edge detection algorithm and its application on GPR[J]. Modern Radar, 2020, 42(8): 41-45. https://www.cnki.com.cn/Article/CJFDTOTAL-XDLD202008008.htm [15] Xydeas C S, Petrovi V. Objective image fusion performance measure[J]. Electronics Letters, 2000, 36(4): 308-309. doi: 10.1049/el:20000267 -
点击查看大图
图(8) / 表(3)
计量
- 文章访问数: 24
- HTML全文浏览量: 11
- PDF下载量: 11
- 被引次数: 0
