Advanced Search
中文
Volume 46 Issue 1
Jan.  2024
Turn off MathJax
Article Contents
Abstract
References
Related Articles
NIU Zhenhua, XING Yanchao, LIN Yingchao, WANG Chenxuan. Infrared and Visible Image Fusion Based on NSCT Combined with Saliency Map and Region Energy[J]. Infrared Technology , 2024, 46(1): 84-93.
Citation: NIU Zhenhua, XING Yanchao, LIN Yingchao, WANG Chenxuan. Infrared and Visible Image Fusion Based on NSCT Combined with Saliency Map and Region Energy[J]. Infrared Technology , 2024, 46(1): 84-93.
shu

Infrared and Visible Image Fusion Based on NSCT Combined with Saliency Map and Region Energy

  • School of Information and Control Engineering, Qingdao University of Technology, Qingdao 266520, China

More Information
  • Received Date: March 21, 2022
  • Revised Date: May 10, 2022
    Graphical Abstract
    • Abstract
  • To address the problems of low clarity and contrast of indistinct targets in traditional infrared and visible image-fusion algorithms, this study proposes a fusion method based on non-subsampled contourlet transform (NSCT) combined with a saliency map and region energy. First, an improved frequency-tuning (FT) method is used to obtain the infrared image saliency map, which is subsequently normalized to obtain the saliency map weight. A single-scale retinex (SSR) algorithm is then used to enhance the visible image. Second, NSCT is used to decompose the infrared and visible images, and a new fusion weight is designed based on the normalized saliency map and region energy to guide low-frequency coefficient fusion. This solves the problem of region-energy adaptive weighting being prone to introducing noise, and the improved "weighted Laplace energy sum" is used to guide the fusion of high-frequency coefficients. Finally, the fused image is obtained by inverse NSCT. Six groups of images were used to compare the proposed method with seven classical methods. The proposed method outperformed others in terms of information entropy, mutual information, average gradient, and standard deviation. Regarding spatial frequency, the first group of images was second best, and the remaining images exhibited the best results. The fusion images displayed rich information, high resolution, high contrast, and moderate brightness, demonstrating suitability for human observation, which verifies the effectiveness of this method.
    • FullText(HTML)
    • References (28)
  • [1]
    杨孙运, 奚峥皓, 王汉东, 等. 基于NSCT和最小化-局部平均梯度的图像融合[J]. 红外技术, 2021, 43(1): 13-20. http://hwjs.nvir.cn/article/id/144252d1-978c-4c1e-85ad-e0b8d5e03bf6

    YANG Sunyun, XI Zhenghao, WANG Handong, et al. Image fusion based on nsct and minimum-local mean gradient[J]. Infrared Technology, 2021, 43(1): 13-20. http://hwjs.nvir.cn/article/id/144252d1-978c-4c1e-85ad-e0b8d5e03bf6
    [2]
    肖中杰. 基于NSCT红外与可见光图像融合算法优化研究[J]. 红外技术, 2017, 39(12): 1127-1130. http://hwjs.nvir.cn/article/id/hwjs201712010

    XIAO Zhongjie. Improved infrared and visible light image fusion algorithm based on NSCT[J]. Infrared Technology, 2017, 39(12): 1127-1130. http://hwjs.nvir.cn/article/id/hwjs201712010
    [3]
    ZHANG B, LU X, PEI H, et al. A fusion algorithm for infrared and visible images based on saliency analysis and non-subsampled Shearlet transform[J]. Infrared Physics & Technology, 2015, 73: 286-297.
    [4]
    ZHOU Z, WANG B, LI S, et al. Perceptual fusion of infrared and visible images through a hybrid multi-scale decomposition with Gaussian and bilateral filters[J]. Information Fusion, 2016, 30: 15-26. DOI: 10.1016/j.inffus.2015.11.003
    [5]
    MA J, YU W, LIANG P, et al. Fusion GAN: A generative adversarial network for infrared and visible image fusion[J]. Information Fusion, 2019, 48: 11-26. DOI: 10.1016/j.inffus.2018.09.004
    [6]
    ZHANG Y, ZHANG L, BAI X, et al. Infrared and visual image fusion through infrared feature extraction and visual information preservation[J]. Infrared Physics & Technology, 2017, 83: 227-237.
    [7]
    Mertens T, Kautz J, Reeth F V. Exposure fusion[C]//Proceedings of Pacific Conference on Computer Graphics and Applications, 2007: 382-390.
    [8]
    ZHANG Z, Blum R S. A categorization of multiscale-decomposition-based image fusion schemes with a performance study for a digital camera application[C]//Proc. IEEE, 1999, 87(8): 1315-1326.
    [9]
    Lewis J J, Callaghan R J O, Nikolov S G, et al. Pixel- and region-based image fusion with complex wavelets[J]. Inf. Fus., 2007, 8(2): 119-130. DOI: 10.1016/j.inffus.2005.09.006
    [10]
    Nencini F, Garzelli A, Baronti S, et al. Remote sensing image fusion using the curvelet transform[J]. Inf. Fus., 2007, 8(2): 143-156. DOI: 10.1016/j.inffus.2006.02.001
    [11]
    YANG S, WANG M, JIAO L, et al. Image fusion based on a new contourlet packet[J]. Inf. Fus., 2010, 11(2): 78-84. DOI: 10.1016/j.inffus.2009.05.001
    [12]
    Cunha A L, ZHOU J P, DO M N. The nonsubsampled Contourlet transform: theory, design, and applications [J]. IEEE Transactions on Image Processing, 2006, 15(10): 3089-3101. DOI: 10.1109/TIP.2006.877507
    [13]
    郭明, 符拯, 奚晓梁. 基于局部能量的NSCT域红外与可见光图像融合算法[J]. 红外与激光工程, 2012, 41(8): 2229-2235.

    GUO Ming, FU Zheng, XI Xiaoliang. Novel fusion algorithm for infrared and visible images based on local energy in NSCT domain[J]. Infrared and Laser Engineering, 2012, 41(8): 2229-2235.
    [14]
    Cands E J, Donoho D L. Curvelets and curvilinear integrals[J]. J. Approximation Theor., 2001, 113(1): 59-90. DOI: 10.1006/jath.2001.3624
    [15]
    DO M N, Vetterli M. Contourlets: a directional multiresolution image representation[C]//Proceedings of IEEE International Conference on Image Processing, 2002, 1: I-357-I-360.
    [16]
    ZHAI Y, SHAH M. Visual attention detection in video se-quencesusing spatiotemporal cues[C]//Proceedings of the 14th ACM Conference on Multimedia, 2006: 815-824.
    [17]
    Achanta R, Hemami S, Estrada F, et al. Frequency-tuned salient region detection[C]// IEEE Conference on Computer Vision & Pattern Recognition, 2009: 1597-1604.
    [18]
    叶坤涛, 李文, 舒蕾蕾, 等. 结合改进显著性检测与NSST的红外与可见光图像融合方法[J]. 红外技术, 2021, 43(12): 1212-1221. http://hwjs.nvir.cn/article/id/bfd9f932-e0bd-4669-b698-b02d42e31805

    YE Kuntao, LI Wen, SHU Leilei, LI Sheng. Infrared and visible image fusion method based on improved saliency detection and non-subsampled shearlet transform[J]. Infrared Technology, 2021, 43(12): 1212-1221. http://hwjs.nvir.cn/article/id/bfd9f932-e0bd-4669-b698-b02d42e31805
    [19]
    唐中剑, 毛春. 基于显著导向的可见光与红外图像融合算法[J]. 太赫兹科学与电子信息学报, 2021, 19(1): 125-131.

    TANG Zhongjian, MAO Chun. Visible and infrared image fusion algorithm based on saliency guidance[J]. Journal of Terahertz Science and Electronic Information Technology, 2021, 19(1): 125-131.
    [20]
    王惠琴, 吕佳芸, 张伟. 基于双边滤波-BM3D算法的GPR图像去噪[J]. 兰州理工大学学报, 2022, 48(1): 91-97.

    WANG Huiqin, LYU Jiayun, ZHANG Wei. GPR image denoising based on bilateral filtering BM3D algorithm[J]. Journal of Lanzhou University of Technology, 2022, 48(1): 91-97.
    [21]
    Edwin H Land. The retinex theory of color vision [J]. Scientific American, 1977, 237(6): 108-129. DOI: 10.1038/scientificamerican1277-108
    [22]
    程铁栋, 卢晓亮, 易其文, 等. 一种结合单尺度Retinex与引导滤波的红外图像增强方法[J]. 红外技术, 2021, 43(11): 1081-1088. http://hwjs.nvir.cn/article/id/b49a0a09-e295-40e6-9736-24a58971206e

    CHENG Tiedong, LU Xiaoliang, YI Qiwen, et al. Research on infrared image enhancement method combined with single-scale retinex and guided image filter[J]. Infrared Technology, 2021, 43(11): 1081-1088. http://hwjs.nvir.cn/article/id/b49a0a09-e295-40e6-9736-24a58971206e
    [23]
    翟海祥, 何嘉奇, 王正家, 等. 改进Retinex与多图像融合算法用于低照度图像增强[J]. 红外技术, 2021, 43(10): 987-993. http://hwjs.nvir.cn/article/id/23500140-4bab-40b3-9bef-282f14dc105e

    ZHAI Haixiang, HE Jiaqi, WANG Zhengjia, et al. Improved Retinex and multi-image fusion algorithm for low illumination image enhancemen[J]. Infrared Technology, 2021, 43(10): 987-993. http://hwjs.nvir.cn/article/id/23500140-4bab-40b3-9bef-282f14dc105e
    [24]
    LIU Y, LIU S, WANG Z. A general framework for image fusion based on multi-scale transform and sparse representation[J]. Information Fusion, 2015, 24: 147-167.
    [25]
    Kumar B. Multifocus and multispectral image fusion based on pixel significance using discrete cosine harmonic wavelet transform[J]. Signal, Image & Video Processing, 2013, 7: 1125-1143. DOI: 10.1007/s11760-012-0361-x
    [26]
    ZHOU Z, BO W, SUN L, et al. Perceptual fusion of infrared and visible images through a hybrid multi-scale decomposition with Gaussian and bilateral filters[J]. Information Fusion, 2016, 30: 15-26.
    [27]
    Toet Alexander. TNO image fusion dataset[EB/OL]. 2014, https://doi.org/10.6084/m9.figshare.1008029.v1.
    [28]
    张小利, 李雄飞, 李军. 融合图像质量评价指标的相关性分析及性能评估[J]. 自动化学报, 2014, 40(2): 306-315.

    ZHANG Xiaoli, LI Xiongfei, LI Jun. Validation and correlation analysis of metrics for evaluating performance of image fusion[J]. Acta Automatica Sinica, 2014, 40(2): 306-315.
    • Related Articles

  • Related Articles

    [1]ZHAO Yating, HAN Long, HE Huihuang, CHEN Chu. DSEL-CNN: Image Fusion Algorithm Combining Attention Mechanism and Balanced Loss[J]. Infrared Technology , 2025, 47(3): 358-366.
    [2]CHEN Zhuang, HE Feng, HONG Xiaohang, ZHANG Qiran, YANG Yuyan. Embedded Platform IR Small-target Detection Based on Self-attention and Convolution Fused Architecture[J]. Infrared Technology , 2025, 47(1): 89-96.
    [3]LI Xu, XIAO Zhiyun, JIANG Yedong, WANG Yazhou, SU Yu. Fault Detection and Identification of Multi-Source Insulators Based on Improved YOLOv7[J]. Infrared Technology , 2024, 46(11): 1325-1333.
    [4]YUE Mingkai, QUAN Kangnan, ZHANG Cong, HAN Ziqiang. Research on Infrared Small Target Detection Algorithm Based on Improved YOLOv8[J]. Infrared Technology , 2024, 46(11): 1286-1292.
    [5]GAO Yongqi, YUAN Zhixiang. Improved YOLOv5-based Underwater Infrared Garbage Detection Algorithm[J]. Infrared Technology , 2024, 46(9): 994-1005.
    [6]WANG You, HAN Lixiang, FU Gui. Aerial Infrared Image Target Recognition Method Based on Improved YOLOv5s[J]. Infrared Technology , 2024, 46(7): 775-781, 801.
    [7]GAO Mingming, LI Yuanzhou, MA Lei, NAN Jingchang, ZHOU Qianyi. YOLOv5-LR: A Rotating Object Detection Model for Remote Sensing Images[J]. Infrared Technology , 2024, 46(1): 43-51.
    [8]SHEN Lingyun, LANG Baihe, SONG Zhengxun, WEN Zhitao. Remote Sensing Image Target Detection Method Based on CSE-YOLOv5[J]. Infrared Technology , 2023, 45(11): 1187-1197.
    [9]KONG Songtao, XU Zhenze, LIN Xingyu, ZHANG Chunqiu, JIANG Guoqing, ZHANG Chunqing, WANG Kun. Infrared Thermal Imaging Defect Detection of Photovoltaic Module Based on Improved YOLO v5 Algorithm[J]. Infrared Technology , 2023, 45(9): 974-981.
    [10]HU Yan, HU Haobing, ZHAO Yuhang, YUAN Zihao, SI Chengke. Infrared Thermal Imaging Low-Resolution and Small Pedestrian Target Detection Method[J]. Infrared Technology , 2022, 44(11): 1146-1153.

  • Cited by

    Periodical cited type(25)

    1. 李阳，丘建培，宋坤. 基于音视频多模态数据感知的智能巡检系统设计与应用. 现代信息科技. 2025(03): 189-193 .
    2. 周亚男. 光伏电站运维现状分析. 太阳能. 2024(01): 12-19 .
    3. 兰金江，曾学仁，方亮，田楠，王志强，刘继江. 基于无人机巡检的光伏缺陷检测与定位. 科技创新与应用. 2024(18): 14-19 .
    4. 任鹏，张哲，于洋. 基于边缘计算的县域分布式光伏智能巡检方法. 吉林电力. 2024(03): 28-31 .
    5. 温建国. 智能无人机红外巡检技术在光伏电站故障诊断中的应用. 中国战略新兴产业. 2024(26): 23-25 .
    6. 侯伟，陈雅，宋承继，刘强锋. 基于改进YOLOv5算法的无人机巡检图像智能识别方法. 微型电脑应用. 2024(09): 26-30+36 .
    7. 杨梅，马建新，陈炳森，赵泽政. 光伏电站无人机自动巡检及故障诊断技术应用. 计量与测试技术. 2024(09): 89-92 .
    8. 吴张宇，吴池莉，于慧铭，政幸男，张啸宇. 面向大规模光伏电站的无人机巡检路径规划策略. 综合智慧能源. 2024(11): 46-53 .
    9. 李峰，林维修，乐锋，许育燕，张斌. 一种基于无人机的光伏异常检测方法研究. 人工智能科学与工程. 2024(04): 86-92 .
    10. 陈大涛，高伟新，宇文磊县，赵良成，高永鑫，吴良，回峰. 基于无人机巡查的光伏电站检查系统设计. 集成电路应用. 2024(12): 72-75 .
    11. 曹瑞安. 基于AI机器视觉技术的新能源无人值守场站自动巡检方法. 电力大数据. 2024(11): 48-56 .
    12. 吕德利，王旋. 一种基于GPS定位技术的无人机智能光伏巡检系统. 科技创新与应用. 2023(06): 37-40 .
    13. 李德维. 光伏电站组件诊断中无人机智能巡检的应用. 光源与照明. 2023(01): 102-105 .
    14. 潘巧波，李昂，何梓瑜，唐梓彭. 数字化电厂智慧平台在光伏电站的应用. 黑龙江电力. 2023(02): 137-142 .
    15. 张永伟，李贵，马玉权，汪海波. 基于高精度快速故障识别的智能光伏视频巡检系统研究. 电力信息与通信技术. 2023(06): 73-78 .
    16. 范群. 智能集控平台在光伏发电站生产中的应用策略. 光源与照明. 2023(06): 142-144 .
    17. 白玉龙，孙茹洁，哈永华. 光伏电站自主巡检中的无人机视觉定位算法研究. 电子元器件与信息技术. 2023(05): 72-75 .
    18. 邓拥正，杨健. 浅谈无人机在光伏电站巡检中的应用. 红水河. 2023(04): 69-72 .
    19. 王佳文，朱永灿，王帅，李科锋. 航拍光伏组件图像的畸变校正方法研究. 湖南电力. 2023(04): 74-79 .
    20. 周登科，郭星辰，史凯特，汤鹏，郑开元，马鹏阁. 风电场无人机巡检红外叶片图像拼接算法. 红外技术. 2023(12): 1161-1168 . 本站查看
    21. 李智强. 基于无人机航拍摄影的变电站运行环境智能巡检方法. 电气技术与经济. 2023(10): 146-148 .
    22. 艾上美，周剑峰，张必朝，张涛，王红斌. 基于改进SSD算法的光伏组件缺陷检测研究. 智慧电力. 2023(12): 53-58 .
    23. 周登科，郭星辰，史凯特，汤鹏，郑开元，马鹏阁. 风电场无人机巡检红外叶片图像拼接算法. 红外技术. 2023(11): 1161-1168 . 本站查看
    24. 孙霞，张洁，赵厚群，张坤乾，缪玉婷. Petri网在架空电缆无人机巡检方面的研究. 绥化学院学报. 2022(12): 139-142 .
    25. 李垚，魏文震，杨增健，赵鑫，吕健. 基于大数据的变电站在线智能巡视系统的研究. 电力大数据. 2022(11): 47-55 .

    Other cited types(10)

Catalog

    Get Citation
    PDF
    XML
    Article views PDF downloads Cited by(35)
    Related

    Copyright © 《Infrared Technology》Editorial Office滇ICP备12000354号-3

    Supported by: Beijing Renhe Information Technology Co., Ltd.  Baidu Analytics

    /

    DownLoad:  Full-Size Img  PowerPoint
    Return
    Return