Advanced Search
中文
Volume 46 Issue 1
Jan.  2024
Turn off MathJax
Article Contents
Abstract
References
Related Articles
NIU Zhenhua, XING Yanchao, LIN Yingchao, WANG Chenxuan. Infrared and Visible Image Fusion Based on NSCT Combined with Saliency Map and Region Energy[J]. Infrared Technology , 2024, 46(1): 84-93.
Citation: NIU Zhenhua, XING Yanchao, LIN Yingchao, WANG Chenxuan. Infrared and Visible Image Fusion Based on NSCT Combined with Saliency Map and Region Energy[J]. Infrared Technology , 2024, 46(1): 84-93.
shu

Infrared and Visible Image Fusion Based on NSCT Combined with Saliency Map and Region Energy

  • School of Information and Control Engineering, Qingdao University of Technology, Qingdao 266520, China

More Information
  • Received Date: March 21, 2022
  • Revised Date: May 10, 2022
    Graphical Abstract
    • Abstract
  • To address the problems of low clarity and contrast of indistinct targets in traditional infrared and visible image-fusion algorithms, this study proposes a fusion method based on non-subsampled contourlet transform (NSCT) combined with a saliency map and region energy. First, an improved frequency-tuning (FT) method is used to obtain the infrared image saliency map, which is subsequently normalized to obtain the saliency map weight. A single-scale retinex (SSR) algorithm is then used to enhance the visible image. Second, NSCT is used to decompose the infrared and visible images, and a new fusion weight is designed based on the normalized saliency map and region energy to guide low-frequency coefficient fusion. This solves the problem of region-energy adaptive weighting being prone to introducing noise, and the improved "weighted Laplace energy sum" is used to guide the fusion of high-frequency coefficients. Finally, the fused image is obtained by inverse NSCT. Six groups of images were used to compare the proposed method with seven classical methods. The proposed method outperformed others in terms of information entropy, mutual information, average gradient, and standard deviation. Regarding spatial frequency, the first group of images was second best, and the remaining images exhibited the best results. The fusion images displayed rich information, high resolution, high contrast, and moderate brightness, demonstrating suitability for human observation, which verifies the effectiveness of this method.
    • FullText(HTML)
    • References (28)
  • [1]
    杨孙运, 奚峥皓, 王汉东, 等. 基于NSCT和最小化-局部平均梯度的图像融合[J]. 红外技术, 2021, 43(1): 13-20. http://hwjs.nvir.cn/article/id/144252d1-978c-4c1e-85ad-e0b8d5e03bf6

    YANG Sunyun, XI Zhenghao, WANG Handong, et al. Image fusion based on nsct and minimum-local mean gradient[J]. Infrared Technology, 2021, 43(1): 13-20. http://hwjs.nvir.cn/article/id/144252d1-978c-4c1e-85ad-e0b8d5e03bf6
    [2]
    肖中杰. 基于NSCT红外与可见光图像融合算法优化研究[J]. 红外技术, 2017, 39(12): 1127-1130. http://hwjs.nvir.cn/article/id/hwjs201712010

    XIAO Zhongjie. Improved infrared and visible light image fusion algorithm based on NSCT[J]. Infrared Technology, 2017, 39(12): 1127-1130. http://hwjs.nvir.cn/article/id/hwjs201712010
    [3]
    ZHANG B, LU X, PEI H, et al. A fusion algorithm for infrared and visible images based on saliency analysis and non-subsampled Shearlet transform[J]. Infrared Physics & Technology, 2015, 73: 286-297.
    [4]
    ZHOU Z, WANG B, LI S, et al. Perceptual fusion of infrared and visible images through a hybrid multi-scale decomposition with Gaussian and bilateral filters[J]. Information Fusion, 2016, 30: 15-26. DOI: 10.1016/j.inffus.2015.11.003
    [5]
    MA J, YU W, LIANG P, et al. Fusion GAN: A generative adversarial network for infrared and visible image fusion[J]. Information Fusion, 2019, 48: 11-26. DOI: 10.1016/j.inffus.2018.09.004
    [6]
    ZHANG Y, ZHANG L, BAI X, et al. Infrared and visual image fusion through infrared feature extraction and visual information preservation[J]. Infrared Physics & Technology, 2017, 83: 227-237.
    [7]
    Mertens T, Kautz J, Reeth F V. Exposure fusion[C]//Proceedings of Pacific Conference on Computer Graphics and Applications, 2007: 382-390.
    [8]
    ZHANG Z, Blum R S. A categorization of multiscale-decomposition-based image fusion schemes with a performance study for a digital camera application[C]//Proc. IEEE, 1999, 87(8): 1315-1326.
    [9]
    Lewis J J, Callaghan R J O, Nikolov S G, et al. Pixel- and region-based image fusion with complex wavelets[J]. Inf. Fus., 2007, 8(2): 119-130. DOI: 10.1016/j.inffus.2005.09.006
    [10]
    Nencini F, Garzelli A, Baronti S, et al. Remote sensing image fusion using the curvelet transform[J]. Inf. Fus., 2007, 8(2): 143-156. DOI: 10.1016/j.inffus.2006.02.001
    [11]
    YANG S, WANG M, JIAO L, et al. Image fusion based on a new contourlet packet[J]. Inf. Fus., 2010, 11(2): 78-84. DOI: 10.1016/j.inffus.2009.05.001
    [12]
    Cunha A L, ZHOU J P, DO M N. The nonsubsampled Contourlet transform: theory, design, and applications [J]. IEEE Transactions on Image Processing, 2006, 15(10): 3089-3101. DOI: 10.1109/TIP.2006.877507
    [13]
    郭明, 符拯, 奚晓梁. 基于局部能量的NSCT域红外与可见光图像融合算法[J]. 红外与激光工程, 2012, 41(8): 2229-2235.

    GUO Ming, FU Zheng, XI Xiaoliang. Novel fusion algorithm for infrared and visible images based on local energy in NSCT domain[J]. Infrared and Laser Engineering, 2012, 41(8): 2229-2235.
    [14]
    Cands E J, Donoho D L. Curvelets and curvilinear integrals[J]. J. Approximation Theor., 2001, 113(1): 59-90. DOI: 10.1006/jath.2001.3624
    [15]
    DO M N, Vetterli M. Contourlets: a directional multiresolution image representation[C]//Proceedings of IEEE International Conference on Image Processing, 2002, 1: I-357-I-360.
    [16]
    ZHAI Y, SHAH M. Visual attention detection in video se-quencesusing spatiotemporal cues[C]//Proceedings of the 14th ACM Conference on Multimedia, 2006: 815-824.
    [17]
    Achanta R, Hemami S, Estrada F, et al. Frequency-tuned salient region detection[C]// IEEE Conference on Computer Vision & Pattern Recognition, 2009: 1597-1604.
    [18]
    叶坤涛, 李文, 舒蕾蕾, 等. 结合改进显著性检测与NSST的红外与可见光图像融合方法[J]. 红外技术, 2021, 43(12): 1212-1221. http://hwjs.nvir.cn/article/id/bfd9f932-e0bd-4669-b698-b02d42e31805

    YE Kuntao, LI Wen, SHU Leilei, LI Sheng. Infrared and visible image fusion method based on improved saliency detection and non-subsampled shearlet transform[J]. Infrared Technology, 2021, 43(12): 1212-1221. http://hwjs.nvir.cn/article/id/bfd9f932-e0bd-4669-b698-b02d42e31805
    [19]
    唐中剑, 毛春. 基于显著导向的可见光与红外图像融合算法[J]. 太赫兹科学与电子信息学报, 2021, 19(1): 125-131.

    TANG Zhongjian, MAO Chun. Visible and infrared image fusion algorithm based on saliency guidance[J]. Journal of Terahertz Science and Electronic Information Technology, 2021, 19(1): 125-131.
    [20]
    王惠琴, 吕佳芸, 张伟. 基于双边滤波-BM3D算法的GPR图像去噪[J]. 兰州理工大学学报, 2022, 48(1): 91-97.

    WANG Huiqin, LYU Jiayun, ZHANG Wei. GPR image denoising based on bilateral filtering BM3D algorithm[J]. Journal of Lanzhou University of Technology, 2022, 48(1): 91-97.
    [21]
    Edwin H Land. The retinex theory of color vision [J]. Scientific American, 1977, 237(6): 108-129. DOI: 10.1038/scientificamerican1277-108
    [22]
    程铁栋, 卢晓亮, 易其文, 等. 一种结合单尺度Retinex与引导滤波的红外图像增强方法[J]. 红外技术, 2021, 43(11): 1081-1088. http://hwjs.nvir.cn/article/id/b49a0a09-e295-40e6-9736-24a58971206e

    CHENG Tiedong, LU Xiaoliang, YI Qiwen, et al. Research on infrared image enhancement method combined with single-scale retinex and guided image filter[J]. Infrared Technology, 2021, 43(11): 1081-1088. http://hwjs.nvir.cn/article/id/b49a0a09-e295-40e6-9736-24a58971206e
    [23]
    翟海祥, 何嘉奇, 王正家, 等. 改进Retinex与多图像融合算法用于低照度图像增强[J]. 红外技术, 2021, 43(10): 987-993. http://hwjs.nvir.cn/article/id/23500140-4bab-40b3-9bef-282f14dc105e

    ZHAI Haixiang, HE Jiaqi, WANG Zhengjia, et al. Improved Retinex and multi-image fusion algorithm for low illumination image enhancemen[J]. Infrared Technology, 2021, 43(10): 987-993. http://hwjs.nvir.cn/article/id/23500140-4bab-40b3-9bef-282f14dc105e
    [24]
    LIU Y, LIU S, WANG Z. A general framework for image fusion based on multi-scale transform and sparse representation[J]. Information Fusion, 2015, 24: 147-167.
    [25]
    Kumar B. Multifocus and multispectral image fusion based on pixel significance using discrete cosine harmonic wavelet transform[J]. Signal, Image & Video Processing, 2013, 7: 1125-1143. DOI: 10.1007/s11760-012-0361-x
    [26]
    ZHOU Z, BO W, SUN L, et al. Perceptual fusion of infrared and visible images through a hybrid multi-scale decomposition with Gaussian and bilateral filters[J]. Information Fusion, 2016, 30: 15-26.
    [27]
    Toet Alexander. TNO image fusion dataset[EB/OL]. 2014, https://doi.org/10.6084/m9.figshare.1008029.v1.
    [28]
    张小利, 李雄飞, 李军. 融合图像质量评价指标的相关性分析及性能评估[J]. 自动化学报, 2014, 40(2): 306-315.

    ZHANG Xiaoli, LI Xiongfei, LI Jun. Validation and correlation analysis of metrics for evaluating performance of image fusion[J]. Acta Automatica Sinica, 2014, 40(2): 306-315.
    • Related Articles

  • Related Articles

    [1]WANG Wenjin, KONG Jincheng, QI Wenbin, ZHANG Yang, SONG Linwei, WU Jun, ZHAO Wen, YU Jianyun, QIN Gang. Research Progress on Materials and Devices of HgCdTe p-on-n Double Layer Heterojunction Grown by VLPE[J]. Infrared Technology , 2024, 46(3): 233-245.
    [2]YANG Chunzhang, QIN Gang, LI Yanhui, LI Da, KONG Jincheng. Research on Growth of M/L-wavelength Dual-band IR-MCT on CZT Substrate by MBE[J]. Infrared Technology , 2018, 40(1): 1-5.
    [3]ZHOU Lianjun, HAN Fuzhong, BAI Piji, SHU Chang, SUN Hao, WANG Xiaojuan, LI Jinghui, ZOU Pengcheng, GUO Jianhua, WANG Qiongfang. Review of HOT MW Infrared Detector Using MCT Technology[J]. Infrared Technology , 2017, 39(2): 116-124.
    [4]QIN Gang, LI Dongsheng, LI Xiongjun, LI Yanhui, WANG Xiangqian, YANG Yan, TIE Xiaoying, ZUO Dafan, BO Junxiang. Research on the Technique of in-situ p-on-n MWIR-MCT by MBE[J]. Infrared Technology , 2016, 38(10): 820-824.
    [5]WANG Yi-feng, LI Pei-zhi, LIU Li-ming, WANG Dan-lin. Developments of Very Long Wavelength Mercury Cadmium Telluride Infrared Detectors[J]. Infrared Technology , 2012, 34(7): 373-382. DOI: 10.3969/j.issn.1001-8891.2012.07.001
    [6]Developments of Mercury Cadmium Telluride in Recent Years[J]. Infrared Technology , 2009, 31(8): 435-442. DOI: 10.3969/j.issn.1001-8891.2009.08.001
    [7]The Determination of Cadmium-Mercury Telluride Composition for Any Thickness by Infrared Transmission[J]. Infrared Technology , 2005, 27(1): 39-41. DOI: 10.3969/j.issn.1001-8891.2005.01.009
    [8]Measurement on Minority Carrier Lifetime of Mercury Cadmium Telluride Material by Microwave Photoconductivity Decay Method[J]. Infrared Technology , 2003, 25(6): 42-44,48. DOI: 10.3969/j.issn.1001-8891.2003.06.012
    [9]p+n Infrared Detectors by As Ion Implantation in HgCdTe[J]. Infrared Technology , 2002, 24(4): 46-48,26. DOI: 10.3969/j.issn.1001-8891.2002.04.012
    [10]The Surface Passivation of MCT Infrared Detectors[J]. Infrared Technology , 2001, 23(3): 9-12,15. DOI: 10.3969/j.issn.1001-8891.2001.03.003

  • Cited by

    Periodical cited type(4)

    1. 王振，刘磊. 基于改进分水岭算法的电力设备红外图像分割. 红外技术. 2025(04): 484-492 . 本站查看
    2. 刘沛津，张香瑞，魏平. 基于融合重构的电气设备红外图像EnFCM聚类分割方法. 红外技术. 2024(03): 295-304 . 本站查看
    3. 张利军. 基于红外热成像技术的变电站巡检机器人的应用. 山东煤炭科技. 2024(12): 168-172 .
    4. 冯杰，张莹，叶影，贺润平，王哲斐. NSST域电气设备红外图像增强处理算法设计. 电子设计工程. 2023(21): 176-179+185 .

    Other cited types(3)

Catalog

    Get Citation
    PDF
    XML
    Article views PDF downloads Cited by(7)
    Related

    Copyright © 《Infrared Technology》Editorial Office滇ICP备12000354号-3

    Supported by: Beijing Renhe Information Technology Co., Ltd.  Baidu Analytics

    /

    DownLoad:  Full-Size Img  PowerPoint
    Return
    Return