CHEN Sijing, FU Zhitao, LI Ziqian, NIE Han, SONG Jiawen. A Visible and Infrared Image Fusion Algorithm Based on Adaptive Enhancement and Saliency Detection[J]. Infrared Technology , 2023, 45(9): 907-914.
Citation: CHEN Sijing, FU Zhitao, LI Ziqian, NIE Han, SONG Jiawen. A Visible and Infrared Image Fusion Algorithm Based on Adaptive Enhancement and Saliency Detection[J]. Infrared Technology , 2023, 45(9): 907-914.

A Visible and Infrared Image Fusion Algorithm Based on Adaptive Enhancement and Saliency Detection

More Information
  • Received Date: June 21, 2022
  • Revised Date: August 09, 2022
  • This paper proposes a visible and infrared image fusion algorithm to solve the problem of the poor visibility of visible images and control the input volume of visible and infrared images. The proposed method combines image adaptive enhancement with uniqueness (U), focus (F), and object (O) saliency detection. First, an adaptive enhancement algorithm was applied to the visible image to improve the visibility of the textural details and normalize the infrared image. Second, the processed image was decomposed into a detail layer and base layer using guided filtering. A weight map of the detail layer was generated using saliency detection to improve the accuracy of the fusion of the background information of the visible image and the edge information of the infrared image in the detail layer. Finally, the fused image was obtained by combining the detail and base layers. To verify the performance of the proposed algorithm, five fusion evaluation indices: image entropy, average gradient, edge intensity, spatial frequency, and visual fidelity, were selected to quantitatively analyze the fused images. The YOLO v5 network was used to perform target detection for each fusion algorithm. The results show that the proposed algorithm achieved the optimal average accuracy in terms of the qualitative, quantitative, and target detection evaluation indexes of fusion.
  • [1]
    ZHANG X C, YE P, XIAO G. VIFB: a visible and infrared image fusion benchmark[C]//2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), 2020: 14-19.
    [2]
    王君尧, 王志社, 武圆圆, 等. 红外与可见光图像多特征自适应融合方法[J]. 红外技术, 2022, 44(6): 571-579. http://hwjs.nvir.cn/article/id/8dfebcac-d211-44a9-935d-54fecd2afa8a

    WANG Junyao, WANG Zhishe, WU Yuanyuan, et al. Multi-feature adaptive fusion method for infrared and visible images[J]. Infrared Technology, 2022, 44(6): 571-579. http://hwjs.nvir.cn/article/id/8dfebcac-d211-44a9-935d-54fecd2afa8a
    [3]
    MA J, MA Y, LI C. Infrared and visible image fusion methods and applications: A survey[J]. Information Fusion, 2019, 45: 153-178. DOI: 10.1016/j.inffus.2018.02.004
    [4]
    ZHOU Z, DONG M, XIE X, et al. Fusion of infrared and visible images for night-vision context enhancement[J]. Applied Optics, 2016, 55(23): 6480. DOI: 10.1364/AO.55.006480
    [5]
    LI S, KANG X, HU J. Image fusion with guided filtering[J]. IEEE Transactions on Image Processing, 2013, 22(7): 2864-2875. DOI: 10.1109/TIP.2013.2244222
    [6]
    谢伟, 周玉钦, 游敏. 融合梯度信息的改进引导滤波[J]. 中国图象图形学报, 2016, 21(9): 1119-1126. https://www.cnki.com.cn/Article/CJFDTOTAL-ZGTB201609001.htm

    XIE W, ZHOU Y Q, YOU M. Improved guided image filtering integrated with gradient information[J]. Journal of Image and Graphics, 2016, 21(9): 1119-1126. https://www.cnki.com.cn/Article/CJFDTOTAL-ZGTB201609001.htm
    [7]
    FAN Z, YAN L, XIA Y, et al. Fusion of multi-resolution visible image and infrared images based on guided filter[C]//Technical Committee on Control Theory, Chinese Association of Automation, 2018: 4449-4454.
    [8]
    Bavirisetti D P, XIAO G, ZHAO J, et al. Multi-scale guided image and video fusion: a fast and efficient approach[J]. Circuits, Systems, and Signal Processing, 2019, 38(12): 5576-5605. DOI: 10.1007/s00034-019-01131-z
    [9]
    叶坤涛, 李文, 舒蕾蕾, 等. 结合改进显著性检测与NSST的红外与可见光图像融合方法[J]. 红外技术, 2021, 43(12): 1212-1221. http://hwjs.nvir.cn/article/id/bfd9f932-e0bd-4669-b698-b02d42e31805

    YE Kuntao, LI Wen, SHU Leilei, et al. Infrared and visible image fusion method based on improved saliency detection and non-subsampled shearlet transform[J]. Infrared Technology, 2021, 43(12): 1212-1221. http://hwjs.nvir.cn/article/id/bfd9f932-e0bd-4669-b698-b02d42e31805
    [10]
    PENG J, LING H, YU J, et al. Salient region detection by UFO: uniqueness, focusness and objectness[C]//IEEE International Conference on Computer Vision. IEEE, 2013: 1976-1983.
    [11]
    Durand F, Dorsey J. Fast bilateral filtering for the display of high-dynamic-range images[J]. ACM Transactions on Graphics, 2002, 21(3): 257-266. DOI: 10.1145/566654.566574
    [12]
    Bavirisetti D P, Kollu V, GANG X, et al. Fusion of MRI and CT images using guided image filter and image statistics[J]. International Journal of Imaging Systems and Technology, 2017, 27(3): 227-237. DOI: 10.1002/ima.22228
    [13]
    HUI L, WU X, Kittler J. Infrared and visible image fusion using a deep learning framework[C]//International Conference on Pattern Recognition, IEEE, 2018: 2705-2710.
    [14]
    MA J, ZHANG H, SHAO Z, et al. GANMcC: a generative adversarial network with multiclassification constraints for infrared and visible image fusion[J]. IEEE Transactions on Instrumentation and Measurement, 2021, 70: 1-14.
    [15]
    TAN W, ZHOU H, SONG J, et al. Infrared and visible image perceptive fusion through multi-level Gaussian curvature filtering image decomposition[J]. Applied Optics, 2019, 58(12): 3064. DOI: 10.1364/AO.58.003064
    [16]
    ZHOU Z, WANG B, LI S, et al. Perceptual fusion of infrared and visible images through a hybrid multi-scale decomposition with Gaussian and bilateral filters[J]. Information Fusion, 2016, 30: 15-26. DOI: 10.1016/j.inffus.2015.11.003
    [17]
    TAN W, Tiwari P, Pandey H M, et al. Multimodal medical image fusion algorithm in the era of big data[J/OL]. Neural Computing & Applications, 2020: 1-21. https://doi.org/10.1007/s00521-020-05173-2.
    [18]
    MA J, ZHOU Z, WANG B, et al. Infrared and visible image fusion based on visual saliency map and weighted least square optimization[J]. Infrared Physics & Technology, 2017, 82: 8-17.
    [19]
    MA J, CHEN C, LI C, et al. Infrared and visible image fusion via gradient transfer and total variation minimization[J]. Information Fusion, 2016, 31: 100-109.
  • Cited by

    Periodical cited type(6)

    1. 林斌,刘亚军,吴燕东,董晋国. 煤矿带式输送机自换电巡检机器人关键技术研究. 煤炭技术. 2025(03): 248-250 .
    2. 常凯旋,黄建华,孙希延,罗键,包世涛,黄焕生. 基于双模态图像融合的无人机光学小目标检测算法. 激光与光电子学进展. 2025(04): 279-293 .
    3. 宋冬梅. 基于模糊数学理论的灰度图像边缘信息智能检测方法. 电子设计工程. 2025(08): 130-135 .
    4. 杨家全,李邦源,丁贞煜,马文龙,汪航,孙宏滨. 基于多重先验的无监督学习红外图像增强算法. 云南电力技术. 2024(02): 33-40 .
    5. 贺养慧. 基于生成对抗网络的可见光和红外图像融合研究. 激光杂志. 2024(10): 120-124 .
    6. 周君,高焱,姜晴. 双边滤波下的低光照激光雷达图像超分辨增强技术. 激光杂志. 2024(12): 131-137 .

    Other cited types(2)

Catalog

    Article views (207) PDF downloads (75) Cited by(8)
    Related

    /

    DownLoad:  Full-Size Img  PowerPoint
    Return
    Return