Infrared and Visible Image Fusion Method Based on Improved Saliency Detection and Non-subsampled Shearlet Transform

YE Kuntao; LI Wen; SHU Leilei; LI Sheng

Volume 43 Issue 12

Dec. 2021

Turn off MathJax

Article Contents

Article Navigation > Infrared Technology > 2021 > 43(12): 1212-1221

YE Kuntao, LI Wen, SHU Leilei, LI Sheng. Infrared and Visible Image Fusion Method Based on Improved Saliency Detection and Non-subsampled Shearlet Transform[J]. Infrared Technology , 2021, 43(12): 1212-1221.

Citation:

YE Kuntao, LI Wen, SHU Leilei, LI Sheng. Infrared and Visible Image Fusion Method Based on Improved Saliency Detection and Non-subsampled Shearlet Transform[J]. Infrared Technology , 2021, 43(12): 1212-1221.

Citation:

PDF( 2100 KB)

Infrared and Visible Image Fusion Method Based on Improved Saliency Detection and Non-subsampled Shearlet Transform

School of Science, Jiangxi University of Science and Technology, Ganzhou 341000, China

Received Date: 2021-04-02
Rev Recd Date: 2021-04-13
Publish Date: 2021-12-20

Abstract

Abstract

To address the problems in the current infrared and visible image fusion method wherein targets are not prominent and contrast is low based on saliency detection, this paper proposes a fusion method by combining improved saliency detection and non-subsampled shearlet transform (NSST). First, the improved maximum symmetric surround algorithm is used to extract the saliency map of an infrared image, the improved gamma correction method is utilized to enhance the map, and the visible image is enhanced through homomorphic filtering. Second, the infrared and enhanced visible images are decomposed into low-and high-frequency parts through NSST, and the saliency map is used to guide the fusion of the low-frequency parts. Simultaneously, the rule of maximum region energy selection is used to guide the fusion of the high-frequency parts. Finally, the fusion image is reconstructed using the inverse NSST. The experimental results show that the proposed method is far superior to other seven fusion methods in terms of average gradient, information entropy, spatial frequency, and standard deviation. Thus, proposed method can effectively highlight the infrared target, improve the contrast and definition of fused images, and preserve rich background information of visible images.
- image fusion,
- saliency detection,
- non-subsampled shearlet transform,
- maximum symmetric surround,
- homomorphicfiltering

FullText(HTML)

References(20)

References

[1]	CHEN J, LI X, LUO L, et al. Infrared and visible image fusion based on target-enhanced multiscale transform decomposition[J]. Information Sciences, 2020, 508(1): 64-78. http://www.sciencedirect.com/science/article/pii/S0020025519308163
[2]	崔晓荣, 沈涛, 黄建鲁, 等. 基于BEMD改进的视觉显著性红外和可见光图像融合[J]. 红外技术, 2020, 42(11): 1061-1071. http://hwjs.nvir.cn/article/id/c89c0447-6d07-4a75-99f6-1bf8681cf588 CUI Xiaorong, SHEN Tao, HUANG Jianlu, et al. Infrared and visble image fusion based on BEMD and improved visual saliency[J]. Infrared Technology, 2020, 42(11): 1061-1071. http://hwjs.nvir.cn/article/id/c89c0447-6d07-4a75-99f6-1bf8681cf588
[3]	CHENG B, JIN L, LI G. Infrared and visual image fusion using LNSST and an adaptive dual-channel PCNN with triple-linking strength[J]. Neurocomputing, 2018, 310(10): 135-147. http://www.onacademic.com/detail/journal_1000040423651710_9d04.html
[4]	郑伟, 赵成晨, 郝冬梅. NSST和改进PCNN相结合的甲状腺图像融合[J]. 光电工程, 2016, 43(10): 42-48, 55. https://www.cnki.com.cn/Article/CJFDTOTAL-GDGC201610008.htm ZHENG Wei, ZHAO Chengchen, HAO Dongmei. Thyroid image fusion based on NSST and improved PCNN[J]. Opto-Electronic Engineering, 2016, 43(10): 42-48, 55. https://www.cnki.com.cn/Article/CJFDTOTAL-GDGC201610008.htm
[5]	Borji A, Itti L. State-of-the-art in visual attention modeling[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2013, 35(1): 185-207. doi: 10.1109/TPAMI.2012.89
[6]	傅志中, 王雪, 李晓峰, 等. 基于视觉显著性和NSCT的红外与可见光图像融合[J]. 电子科技大学学报, 2017, 46(2): 357-362. doi: 10.3969/j.issn.1001-0548.2017.02.007 FU Zhizhong, WANG Xue, LI Xiaofeng, et al. Infrared and visible image fusion based on visual saliency and NSCT[J]. Journal of University of Electronic Science and Technology of China, 2017, 46(2): 357-363. doi: 10.3969/j.issn.1001-0548.2017.02.007
[7]	林子慧, 魏宇星, 张建林, 等. 基于显著性图的红外与可见光图像融合[J]. 红外技术, 2019, 41(7): 640-645. http://hwjs.nvir.cn/article/id/hwjs201907008 LIN Zihui, WEI Yuxing, ZHANG Jianlin, et al. Image fusion of infrared and visible images based on saliency map[J]. Infrared Technology, 2019, 41(7): 640-645. http://hwjs.nvir.cn/article/id/hwjs201907008
[8]	安影, 范训礼, 陈莉, 等. 结合FABEMD和改进的显著性检测的图像融合[J]. 系统工程与电子技术, 2020, 42(2): 292-300. https://www.cnki.com.cn/Article/CJFDTOTAL-XTYD202002006.htm AN Ying, FAN Xunli, CHEN Li, et al. Image fusion combining with FABEMD and improved saliency detection[J]. Systems Engineering and Electronics, 2020, 42(2): 292-300. https://www.cnki.com.cn/Article/CJFDTOTAL-XTYD202002006.htm
[9]	Achanta R, Hemami S, Estrada F, et al. Frequency-tuned salient region detection[C]//IEEE Conference on Computer Vision & Pattern Recognition, 2009: 1597-1604.
[10]	Achanta R, Susstrunk S. Saliency detection using maximum symmetric surround[C]//Proc. of IEEE International Conference on Image Processing, 2010: 2653-2656.
[11]	冯维, 吴贵铭, 赵大兴, 等. 多图像融合Retinex用于弱光图像增强[J]. 光学精密工程, 2020, 28(3): 736-744. https://www.cnki.com.cn/Article/CJFDTOTAL-GXJM202003024.htm FENG Wei, WU Guiming, ZHAO Daxing, et al. Multi images fusion Retinex for low light image enhancement[J]. Optics and Precision Engineering, 2020, 28(3): 736-744. https://www.cnki.com.cn/Article/CJFDTOTAL-GXJM202003024.htm
[12]	吴京辉, 唐林波, 赵保军. 局部特征与全局信息联合的自适应图像增强算法[J]. 北京理工大学学报, 2014, 34(9): 955-960. https://www.cnki.com.cn/Article/CJFDTOTAL-BJLG201409015.htm WU Jinghui, TANG Linbo, ZHAO Baojun. Local-to-global adaptive image enhancement algorithm[J]. Transactions of Beijing Institute of Technology, 2014, 34(9): 955-960. https://www.cnki.com.cn/Article/CJFDTOTAL-BJLG201409015.htm
[13]	ZHAI Y, SHAH M. Visual attention detection in video sequences using spatiotemporal cues[C]//Proceedings of the 14th ACM International Conference on Multimedia, 2006: 815-824.
[14]	DONG S, MA J, SU Z, et al. Robust circular marker localization under non-uniform illuminations based on homomorphic filtering[J]. Measurement, 2021, 170(1): 108700. http://www.sciencedirect.com/science/article/pii/S0263224120312070
[15]	焦姣, 吴玲达, 于少波, 等. 混合多尺度分析和改进PCNN相结合的图像融合方法[J]. 计算机辅助设计与图形学学报, 2019, 31(6): 988-996. https://www.cnki.com.cn/Article/CJFDTOTAL-JSJF201906015.htm JIAO Jiao, WU Lingda, YU Shaobo, et al. Image fusion method using multi-scale analysis and improved PCNN[J]. Journal of Computer-Aided Design & Computer Graphics, 2019, 31(6): 988-996. https://www.cnki.com.cn/Article/CJFDTOTAL-JSJF201906015.htm
[16]	Burt P J, Adelson E H. The laplacian pyramid as a compact image code[J]. IEEE Transactions on Communications, 1983, 31(4): 532-540. doi: 10.1109/TCOM.1983.1095851
[17]	Nencini F, Garzelli A, Baronti S, et al. Remote sensing image fusion using the curvelet transform[J]. Information Fusion, 2007, 8(2): 143-156. doi: 10.1016/j.inffus.2006.02.001
[18]	ZHANG Q, GUO B. Multifocus image fusion using the nonsubsampled contourlet transform[J]. Signal Process, 2009, 89(7): 1334-1346. doi: 10.1016/j.sigpro.2009.01.012
[19]	YIN M, LIU X, LIU Y, et al. Medical image fusion with parameter-adaptive pulse coupled neural network in nonsubsampled shearlet transform domain[J]. IEEE Transactions on Instrumentation and Measurement, 2019, 68(1): 49-64. doi: 10.1109/TIM.2018.2838778
[20]	Figshare. TNO Image Fusion Dataset[EB/OL]. [2014-04-26]. https://figshare.com/articles/TNO_Image_Fusion_Dataset/1008029.