Infrared and Visible Image Fusion Based on BEMD and Improved Visual Saliency

CUI Xiaorong; SHEN Tao; HUANG Jianlu; WANG Di

Volume 42 Issue 11

Nov. 2020

Turn off MathJax

Article Contents

Article Navigation > Infrared Technology > 2020 > 42(11): 1061-1071

CUI Xiaorong, SHEN Tao, HUANG Jianlu, WANG Di. Infrared and Visible Image Fusion Based on BEMD and Improved Visual Saliency[J]. Infrared Technology , 2020, 42(11): 1061-1071.

Citation:

CUI Xiaorong, SHEN Tao, HUANG Jianlu, WANG Di. Infrared and Visible Image Fusion Based on BEMD and Improved Visual Saliency[J]. Infrared Technology , 2020, 42(11): 1061-1071.

CUI Xiaorong, SHEN Tao, HUANG Jianlu, WANG Di. Infrared and Visible Image Fusion Based on BEMD and Improved Visual Saliency[J]. Infrared Technology , 2020, 42(11): 1061-1071.

Citation:

CUI Xiaorong, SHEN Tao, HUANG Jianlu, WANG Di. Infrared and Visible Image Fusion Based on BEMD and Improved Visual Saliency[J]. Infrared Technology , 2020, 42(11): 1061-1071.

PDF( 2987 KB)

Infrared and Visible Image Fusion Based on BEMD and Improved Visual Saliency

Rocket Force University of Engineering, Xi'an 710025, China

Received Date: 2020-01-23
Rev Recd Date: 2020-10-28
Publish Date: 2020-11-20

Abstract

Abstract

Aiming at the problems of low target contrast and insufficiently clear images in the process of visual saliency fusion, this paper proposes an improved frequency Tuned algorithm based on bi-dimensional empirical mode decomposition (BEMD). First, the strong points and contour information of the infrared image captured by BEMD is used to guide the generation of saliency maps of the infrared image. Then, the visible image and the enhanced infrared image are subjected to a non-subsampled contourlet transform(NSCT). The saliency map-guided fusion rule is used for the low-frequency part. The high-frequency part is used to set the area energy to be large and rely on the threshold value rules. Finally, the inverse NSCT transform is used to generate a fused image and subjective visual and objective index evaluations are performed to it. The results show that the method in this paper achieves a multi-level and adaptive analysis of the original image, and achieves good vision compared to the contrast methods.
- images fusion,
- bidimensional empirical mode decomposition,
- saliency map,
- nonsubsampled contourlet transform

FullText(HTML)

References(27)

References

[1]	MA C, MIAO Z, ZHANG X P, et al. A saliency prior context model for real-time object tracking[J]. IEEE Transactions on Multimedia, 2017, 19(11): 2415-2424. doi: 10.1109/TMM.2017.2694219
[2]	HU W, YANG Y, ZHANG W, et al. Moving Object Detection Using Tensor Based Low-Rank and Saliently Fused-Sparse Decomposition[J]. IEEE Transactions on Image Processing, 2016, 7149(c): 1-1. http://www.ncbi.nlm.nih.gov/pubmed/27849530
[3]	Da Cunha A L, ZHOU Jianping, Do M N. The nonsubsampled contourlet transform: theory, design, and applications[J]. IEEE Transactions on Image Processing, 2006, 15(10): 3089-3101. doi: 10.1109/TIP.2006.877507
[4]	KONG W, ZHANG L, LEI Y. Novel fusion method for visible light and infrared images based on nsst-sf-pcnn[J]. Infrared Physics & Technology , 2014, 65: 103-112. doi: 10.1007/BF03346396
[5]	BHUIYANS M A, ADHAMI R R, KHAN J F. Fast and Adaptive bidimensional empirical mode decomposition using order-statistics filter based envelope estimation[J]. Eurasip Journal on Advances in Signal Processing, 2008(1): 1-18. doi: 10.1155/2008/728356
[6]	A Toet. Computational versus psychophysical bottom-up image saliency: A comparative evaluation study[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence , 2011, 33(11): 2131-2146. doi: 10.1109/TPAMI.2011.53
[7]	HAREL J, KOCH C, PERONA P. Graph-based visual saliency[C]// Advances in neural information processing systems, 2006: 545-552.
[8]	HOU X, ZHANG L. Saliency detection: A spectral residual approach[C]// Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2007: 1–8.
[9]	ACHANTA R, HEMAMI S, ESTRADA F, et al.Frequency-tuned salient region detection[C]//2009 IEEE Conference on Computer Vision and Pattern Recognition, 2009: 1597-1604.
[10]	傅志中, 王雪, 李晓峰, 等.基于视觉显著性和NSCT的红外与可见光图像融合[J].电子科技大学学报, 2017, 46(2): 357-363. http://d.wanfangdata.com.cn/Periodical/dzkjdxxb201702007 FU Zhizhong, WANG Xue, LI Xiaofeng, et al. Infrared and Visible Image Fusion Based on Visual Saliency and NSCT[J]. Journal of University of Electronic Science and Technology of China, 2017, 46(2): 357-363. http://d.wanfangdata.com.cn/Periodical/dzkjdxxb201702007
[11]	林子慧, 魏宇星, 张建林, 等.基于显著性图的红外与可见光图像融合[J].红外技术, 2019, 41(7): 640-646. http://www.cnki.com.cn/Article/CJFDTotal-HWJS201907009.htm LIN Zihui, WEI Yuxin, ZHANG Jianlin, et al. Image Fusion of Infrared and Visible Images Based on Saliency Map[J]. Infrared Technology, 2019, 41(7): 640-646. http://www.cnki.com.cn/Article/CJFDTotal-HWJS201907009.htm
[12]	安影, 范训礼, 陈莉, 等.结合FABEMD和改进的显著性检测的图像融合[J].系统工程与电子技术, 2020, 42(2): 292-300. http://d.wanfangdata.com.cn/periodical/xtgcydzjs202002006 AN Ying, FAN Xunli, CHEN Li, et al. Image fusion combining with FABEMD and improved saliency detection[J]. Systems Engineering and Electronics, 2020, 42(2): 292-300. http://d.wanfangdata.com.cn/periodical/xtgcydzjs202002006
[13]	Nunes J C, Bouaoune Y, Delechelle E, et al. Image analysis by bidimensional empirical mode decomposition[J]. Image & Vision Computing, 2003, 21(12): 1019-1026. http://www.sciencedirect.com/science/article/pii/S0262885603000945
[14]	HUANG N E, SHEN Z, LONG S R, et al. The empirical mode de-composition and the Hilbert spectrum for nonlinear and non-stationary time series analysis[J]. Proceedings of the Royal Society A: Mathematical, Physical and Engineering Sciences, 1998, 454(1971): 903-995. doi: 10.1098/rspa.1998.0193
[15]	Bidimensional empirical model decomposition method for image processing in sensing system[J]. Computers and Electrical Engineering, 2018, 68: 215 -224.
[16]	王笛, 沈涛, 孙宾宾, 等.基于大气灰度因子的红外图像增强算法[J].激光与红外, 2019, 49(9): 1135-1140. http://kns.cnki.net/KCMS/detail/detail.aspx?dbcode=CJFD&filename=JGHW201909019 WANG Di, SHEN Tao, SUN Binbin, et al. Infrared image enhancement algorithm based on atmospheric gray factor[J]. Laser & Infrared, 2019, 49(9): 1135-1140. http://kns.cnki.net/KCMS/detail/detail.aspx?dbcode=CJFD&filename=JGHW201909019
[17]	MA J, ZHOU Z, WANG B, et al. Infrared and visible image fusion based on visual saliency map and weighted least square optimization[J]. Infrared Physics & Technology, 2017, 82: 8-17. http://www.sciencedirect.com/science/article/pii/S1350449516305928
[18]	ZHAI Y, SHAH M. Visual attention detection in video sequences using spatiotemporal cues[C]//Proceedings of the 14th Annual ACM International Conference on Multimedia, 2006: 815-824.
[19]	CHENG Mingming, Niloy J Mitra, HUANG Xiaolei, et al. Global Contrast based Salient Region Detection[J].IEEE Transactions on Pattern Analysis and Machine Intelligence, 2015, 37: 569-582. http://ieeexplore.ieee.org/document/6871397
[20]	Achanta R, Estrada F, Wils P, et al. Salient region detection and segmentation[C]//International Conference on Computer Vision Systems, 2008: 66-75.
[21]	杨爱萍, 王海新, 王金斌, 等.基于透射率融合与多重导向滤波的单幅图像去雾[J].光学学报, 2019, 38(12): 104-114. http://www.cnki.com.cn/Article/CJFDTotal-GXXB201812014.htm YANG Aiping, WANG Haixin, WANG Jinbin, et al. Image Dehazing Based on Transmission Fusion and Multi-Guided Filtering[J]. Acta Optica Sinica, 2019, 38(12): 104-114. http://www.cnki.com.cn/Article/CJFDTotal-GXXB201812014.htm
[22]	CUI G, FENG H, XU Z, et al. Detail preserved usion of visible and infrared images using regional saliency extraction and multi-scale image decomposition[J]. Optics Communications, 2015, 341: 199-209. doi: 10.1016/j.optcom.2014.12.032
[23]	Piella G, Heijmans H. A new quality metric for image fusion[C]// Proceedings of the International Conference on Image Processing, 2003: 173-176.
[24]	QU G, ZHANG D, YAN P. Information measure for performance of image fusion[J]. Electronics Letters, 2002, 38(7): 313-315. doi: 10.1049/el:20020212
[25]	WANG Z, A C Bovik, H R Sheikh, et al. Image quality assessment: from error visibility to structural similarity[J]. IEEE Transactions on Image Processing, 2004, 13(4): 600-612. http://jamia.bmj.com/external-ref?access_num=10.1109/TIP.2003.819861&link_type=DOI
[26]	Xydeas CS, Petrovic V. Objective image fusion performance measure[J]. Electronics Letters, 2000, 36(4): 308-309. doi: 10.1049/el:20000267
[27]	J W Roberts, J Van Aardt, F Ahmed. Assessment of image fusion procedures using entropy, image quality, and multispectral classification[J]. Journal of Applied Remote Sensing, 2008, 2(1): 023522. doi: 10.1117/1.2945910