Advanced Search
中文
Volume 44 Issue 5
May  2022
Turn off MathJax
Article Contents
Abstract
References
XU Yunying, YANG Rui, HE Tianfu, LIU Shangwei, FAN Tairan, XU Chenchen. Local Fusion Algorithm of Infrared and Visible Light Images Based on Double-Branch Convolutional Neural Network[J]. Infrared Technology , 2022, 44(5): 521-528.
Citation: XU Yunying, YANG Rui, HE Tianfu, LIU Shangwei, FAN Tairan, XU Chenchen. Local Fusion Algorithm of Infrared and Visible Light Images Based on Double-Branch Convolutional Neural Network[J]. Infrared Technology , 2022, 44(5): 521-528.
shu

Local Fusion Algorithm of Infrared and Visible Light Images Based on Double-Branch Convolutional Neural Network

  • School of Electronic Engineering, Jiangsu Ocean University, Lianyungang 222005, China

More Information
  • Received Date: September 22, 2021
  • Revised Date: May 02, 2022
    Graphical Abstract
    • Abstract
  • Both infrared and visible images have certain limitations, and relying on individual types of images cannot meet the practical needs of engineering. Instead, high-quality fused images can be obtained by introducing image fusion techniques. To better guarantee the diversity of the output information features, this study introduces a dual-branch convolutional neural network to achieve local fusion of infrared and visible images. Based on the dual-branch convolutional neural network, red and blue features are obtained from infrared images and visible light images simultaneously, thereby increasing the amount of information in the fusion image. The integer wavelet transform method is used for image compression. When the color-space model is built, the value of the t-factor is adjusted to obtain an ideal fusion image. The experimental results show that the edge information of the image after the fusion of this method is fully preserved, image detail information is enhanced, and fusion effect of infrared and visible images is improved, compared with the existing methods.
    • FullText(HTML)
    • References (28)
  • [1]
    朱莹, 夏亦犁, 裴文江. 基于改进的BEMD的红外与可见光图像融合方法[J]. 计算机科学, 2020, 47(3): 124-129. https://www.cnki.com.cn/Article/CJFDTOTAL-JSJA202003022.htm

    ZHU Ying, XIA Yili, PEI Wenjiang. Infrared and visible image fusion method based on improved BEMD[J]. Computer Science, 2020, 47(3): 124-129. https://www.cnki.com.cn/Article/CJFDTOTAL-JSJA202003022.htm
    [2]
    苏金凤, 张贵仓, 汪凯. 结合鲁棒主成分分析和非下采样轮廓波变换的红外与可见光图像的压缩融合[J]. 激光与光电子学进展, 2020, 57(4): 84-93. https://www.cnki.com.cn/Article/CJFDTOTAL-JGDJ202004008.htm

    SU Jinfeng, ZHANG Guicang, WANG Kai. Compression and fusion of infrared and visible images combined with robust principal component analysis and non-subsampled contourlet transform[J]. Advances in Lasers and Optoelectronics, 2020, 57(4): 84-93. https://www.cnki.com.cn/Article/CJFDTOTAL-JGDJ202004008.htm
    [3]
    邓秋菊, 王宁. 非下采样轮廊波变换的红外与可见光图像融合[J]. 激光杂志, 2020, 41(4): 92-95. https://www.cnki.com.cn/Article/CJFDTOTAL-JGZZ202004018.htm

    DENG Qiuju, WANG Ning. Infrared and visible image fusion based on non-subsampling contour wave transform[J]. Laser Journal, 2020, 41(4): 92-95. https://www.cnki.com.cn/Article/CJFDTOTAL-JGZZ202004018.htm
    [4]
    王兴龙, 朱芳. 基于双通道PCNN的NSST域红外与可见光图像融合[J]. 平顶山学院学报, 2020, 35(2): 55-61. DOI: 10.3969/j.issn.1673-1670.2020.02.011

    WANG Xinglong, ZHU Fang. Fusion of infrared and visible light images in NSST domain based on dual-channel PCNN[J]. Journal of Pingdingshan University, 2020, 35(2): 55-61. DOI: 10.3969/j.issn.1673-1670.2020.02.011
    [5]
    陈潮起, 孟祥超, 邵枫, 等. 一种基于多尺度低秩分解的红外与可见光图像融合方法[J]. 光学学报, 2020, 40(11): 72-80. https://www.cnki.com.cn/Article/CJFDTOTAL-GXXB202011008.htm

    CHEN Chaoqi, MENG Xiangchao, SHAO Feng, et al. A fusion method of infrared and visible light images based on multi-scale low-rank decomposition[J]. Acta Optics Sinica, 2020, 40(11): 72-80. https://www.cnki.com.cn/Article/CJFDTOTAL-GXXB202011008.htm
    [6]
    沈瑜, 陈小朋, 苑玉彬, 等. 基于显著矩阵与神经网络的红外与可见光图像融合[J]. 激光与光电子学进展, 2020, 57(20): 76-86. https://www.cnki.com.cn/Article/CJFDTOTAL-JGDJ202020009.htm

    SHEN Yu, CHEN Xiaopeng, YUAN Yubin, et al. Fusion of infrared and visible light images based on saliency matrix and neural network[J]. Advances in Lasers and Optoelectronics, 2020, 57(20): 76-86. https://www.cnki.com.cn/Article/CJFDTOTAL-JGDJ202020009.htm
    [7]
    张贵仓, 苏金凤, 拓明秀. DTCWT域的红外与可见光图像融合算法[J]. 计算机工程与科学, 2020, 42(7): 1226-1233. DOI: 10.3969/j.issn.1007-130X.2020.07.011

    ZHANG Guicang, SU Jinfeng, TUO Mingxiu. Infrared and visible light image fusion algorithm in DTCWT domain[J]. Computer Engineering and Science, 2020, 42(7): 1226-1233. DOI: 10.3969/j.issn.1007-130X.2020.07.011
    [8]
    任立成, 刘勇, 张建林, 等. 基于特征融合的双分支模板更新跟踪算法[J]. 国外电子测量技术, 2021, 40(5): 14-21. https://www.cnki.com.cn/Article/CJFDTOTAL-GWCL202105006.htm

    REN Licheng, LIU Yong, ZHANG Jianlin, et al. Two-branch template update tracking algorithm based on feature fusion[J]. Foreign Electronic Measurement Technology, 2021, 40(5): 14-21. https://www.cnki.com.cn/Article/CJFDTOTAL-GWCL202105006.htm
    [9]
    吴佼华, 杨学志, 方帅, 等. 基于双分支卷积神经网络的SAR与多光谱图像融合实验[J]. 地理与地理信息科学, 2021, 37(2): 22-30. DOI: 10.3969/j.issn.1672-0504.2021.02.004

    WU Jiaohua, YANG Xuezhi, FANG Shuai, et al. Experiment of SAR and multispectral image fusion based on double-branch convolutional neural network[J]. Geography and Geographic Information Science, 2021, 37(2): 22-30. DOI: 10.3969/j.issn.1672-0504.2021.02.004
    [10]
    薛盼盼, 刘云, 李辉, 等. 基于时域扩张残差网络和双分支结构的人体行为识别[J/OL]. 控制与决策: 1-10[2021-09-29]. https://doi.org/10.13195/j.kzyjc.2021.0648.

    XUE Panpan, LIU Yun, LI Hui, et al. Human Behavior Recognition Based on Time Domain Dilated Residual Network and Dual Branch Structure [J/OL]. Control and Decision: 1-10. [2021-09-29]. https://doi.org/10.13195/j.kzyjc.2021.0648.
    [11]
    张雪, 孟令灿, 聂秀山. 基于双分支网络特征融合的车辆重识别方法[J]. 数据采集与处理, 2021, 36(3): 468-476. https://www.cnki.com.cn/Article/CJFDTOTAL-SJCJ202103005.htm

    ZHANG Xue, MENG Lingcan, NIE Xiushan. Vehicle re-identification method based on dual-branch network feature fusion[J]. Data Collection and Processing, 2021, 36(3): 468-476. https://www.cnki.com.cn/Article/CJFDTOTAL-SJCJ202103005.htm
    [12]
    贺双龙, 杨斌. 基于局部结构和视觉显著特征的红外和可见光图像泊松重构融合算法[J]. 南华大学学报: 自然科学版, 2020, 34(5): 62-70, 76. https://www.cnki.com.cn/Article/CJFDTOTAL-ZNGB202005011.htm

    HE Shuanglong, YANG Bin. Poisson reconstruction fusion algorithm for infrared and visible light images based on local structure and visual salient features[J]. Journal of University of South China: Natural Science Edition, 2020, 34(5): 62-70, 76. https://www.cnki.com.cn/Article/CJFDTOTAL-ZNGB202005011.htm
    [13]
    周哓玲, 江泽涛. 结合脉冲耦合神经网络与引导滤波的红外与可见光图像融合[J]. 光学学报, 2019, 39(11): 132-139. https://www.cnki.com.cn/Article/CJFDTOTAL-GXXB201911016.htm

    ZHOU Weiling, JIANG Zetao. Fusion of infrared and visible light images combined with pulse-coupled neural network and guided filtering[J]. Journal of Optics, 2019, 39(11): 132-139. https://www.cnki.com.cn/Article/CJFDTOTAL-GXXB201911016.htm
    [14]
    刘明君, 董增寿. 基于多特征的红外与可见光图像融合[J]. 激光杂志, 2019, 40(10): 81-85. https://www.cnki.com.cn/Article/CJFDTOTAL-JGZZ201910017.htm

    LIU Mingjun, DONG Zengshou. Fusion of infrared and visible light images based on multi-features[J]. Laser Journal, 2019, 40(10): 81-85. https://www.cnki.com.cn/Article/CJFDTOTAL-JGZZ201910017.htm
    [15]
    王宁, 周铭, 杜庆磊. 一种红外可见光图像融合及其目标识别方法[J]. 空军预警学院学报, 2019, 33(5): 328-332. DOI: 10.3969/j.issn.2095-5839.2019.05.004

    WANG Ning, ZHOU Ming, DU Qinglei. An infrared and visible light image fusion and its target recognition method[J]. Journal of Air Force Early Warning Academy, 2019, 33(5): 328-332. DOI: 10.3969/j.issn.2095-5839.2019.05.004
    [16]
    宫睿, 王小春. BEMD分解和W变换相结合的红外与可见光图像融合[J]. 中国图象图形学报, 2019, 24(6): 987-999. https://www.cnki.com.cn/Article/CJFDTOTAL-ZGTB201906014.htm

    GONG Rui, WANG Xiaochun. Fusion of infrared and visible light images combined with BEMD decomposition and W transform[J]. Chinese Journal of Image and Graphics, 2019, 24(6): 987-999. https://www.cnki.com.cn/Article/CJFDTOTAL-ZGTB201906014.htm
    [17]
    张雷雷, 王铎. 基于局部能量和邻域信息的红外与可见光图像融合[J]. 光电技术应用, 2019, 34(3): 19-22. DOI: 10.3969/j.issn.1673-1255.2019.03.005

    ZHANG Leilei, WANG Duo. Fusion of infrared and visible light images based on local energy and neighborhood information[J]. Application of Optoelectronic Technology, 2019, 34(3): 19-22. DOI: 10.3969/j.issn.1673-1255.2019.03.005
    [18]
    荣传振, 贾永兴, 吴城, 等. 红外与可见光图像分解与融合方法研究[J]. 数据采集与处理, 2019, 34(1): 146-156. https://www.cnki.com.cn/Article/CJFDTOTAL-SJCJ201901015.htm

    RONG Chuanzhen, JIA Yongxing, WU Cheng, et al. Research on infrared and visible image decomposition and fusion methods[J]. Data Acquisition and Processing, 2019, 34(1): 146-156. https://www.cnki.com.cn/Article/CJFDTOTAL-SJCJ201901015.htm
    [19]
    冯鑫. Tetrolet框架下红外与可见光图像融合[J]. 光子学报, 2019, 48(2): 76-84. https://www.cnki.com.cn/Article/CJFDTOTAL-GZXB201902010.htm

    FENG Xin. Fusion of infrared and visible light images under Tetrolet framework[J]. Acta Photonica Sinica, 2019, 48(2): 76-84. https://www.cnki.com.cn/Article/CJFDTOTAL-GZXB201902010.htm
    [20]
    朱平哲. 基于DCT与PSO的可见光与红外图像融合方法[J]. 新疆大学学报: 自然科学版, 2018, 35(4): 452-458. https://www.cnki.com.cn/Article/CJFDTOTAL-XJDZ201804011.htm

    ZHU Pingzhe. Visible light and infrared image fusion method based on DCT and PSO[J]. Journal of Xinjiang University: Natural Science Edition, 2018, 35(4): 452-458. https://www.cnki.com.cn/Article/CJFDTOTAL-XJDZ201804011.htm
    [21]
    孔玲君, 张志华, 曾茜, 等. 基于NSST和SWT的红外与可见光图像融合算法研究[J]. 包装工程, 2018, 39(19): 216-222. https://www.cnki.com.cn/Article/CJFDTOTAL-BZGC201819038.htm

    KONG Lingjun, ZHANG Zhihua, ZENG Qian, et al. Research on infrared and visible light image fusion algorithm based on NSST and SWT[J]. Packaging Engineering, 2018, 39(19): 216-222. https://www.cnki.com.cn/Article/CJFDTOTAL-BZGC201819038.htm
    [22]
    邱泽敏. 结合区域与边缘特征的红外与可见光图像融合算法[J]. 红外技术, 2018, 40(5): 449-454. http://hwjs.nvir.cn/article/id/hwjs201805008

    QIU Zemin. Infrared and visible image fusion algorithm combining region and edge features[J]. Infrared Technology, 2018, 40(5): 449-454. http://hwjs.nvir.cn/article/id/hwjs201805008
    [23]
    朱亚辉. 聚类分析的红外与可见光融合质量指标相似性研究[J]. 计算机系统应用, 2018, 27(2): 216-222. DOI: 10.3969/j.issn.1003-3254.2018.02.037

    ZHU Yahui. Research on the similarity of infrared and visible light fusion quality indicators based on cluster analysis[J]. Computer System Applications, 2018, 27(2): 216-222. DOI: 10.3969/j.issn.1003-3254.2018.02.037
    [24]
    吴冬鹏, 毕笃彦, 马时平, 等. 边缘和对比度增强的NSST域红外与可见光图像融合[J]. 红外技术, 2017, 39(4): 358-364. http://hwjs.nvir.cn/article/id/hwjs201704011

    WU Dongpeng, BI Duyan, MA Shiping, et al. Edge and contrast enhanced fusion of NSST domain infrared and visible light images[J]. Infrared Technology, 2017, 39(4): 358-364. http://hwjs.nvir.cn/article/id/hwjs201704011
    [25]
    傅志中, 王雪, 李晓峰, 等. 基于视觉显著性和NSCT的红外与可见光图像融合[J]. 电子科技大学学报, 2017, 46(2): 357-362. https://www.cnki.com.cn/Article/CJFDTOTAL-DKDX201702007.htm

    FU Zhizhong, WANG Xue, LI Xiaofeng, et al. Fusion of infrared and visible light images based on visual saliency and NSCT[J]. Journal of University of Electronic Science and Technology of China, 2017, 46(2): 357-362. https://www.cnki.com.cn/Article/CJFDTOTAL-DKDX201702007.htm
    [26]
    李昌兴, 王志强. 基于视觉特性的红外与可见光图像融合[J]. 西安邮电大学学报, 2016, 21(6): 98-102. https://www.cnki.com.cn/Article/CJFDTOTAL-XAYD201606017.htm

    LI Changxing, WANG Zhiqiang. Fusion of infrared and visible light images based on visual characteristics[J]. Journal of Xi'an University of Posts and Telecommunications, 2016, 21(6): 98-102. https://www.cnki.com.cn/Article/CJFDTOTAL-XAYD201606017.htm
    [27]
    周华兵, 侯积磊, 吴伟, 等. 基于语义分割的红外和可见光图像融合[J]. 计算机研究与发展, 2021, 58(2): 436-443. https://www.cnki.com.cn/Article/CJFDTOTAL-JFYZ202102020.htm

    ZHOU Huabing, HOU Jilei, WU Wei, et al. Fusion of infrared and visible light images based on semantic segmentation[J]. Journal of Computer Research and Development, 2021, 58(2): 436-443. https://www.cnki.com.cn/Article/CJFDTOTAL-JFYZ202102020.htm
    [28]
    张蕾. 采用改进平均梯度与自适应PCNN的图像融合[J]. 计算机应用与软件, 2021, 38(3): 218-223. https://www.cnki.com.cn/Article/CJFDTOTAL-JYRJ202103034.htm

    ZHANG Lei. Image fusion using improved average gradient and adaptive PCNN[J]. Computer Applications and Software, 2021, 38(3): 218-223. https://www.cnki.com.cn/Article/CJFDTOTAL-JYRJ202103034.htm
    • Related Articles

  • Cited by

    Periodical cited type(3)

    1. 张健，黄安穴. 基于划区域宇宙算法的红外与可见光图像融合研究. 光电子·激光. 2024(09): 962-970 .
    2. 唐泽恬，童林，龙先梅. 基于CNN与直方图规定化的红外与低照度可见光图像融合. 计算技术与自动化. 2023(03): 102-106+146 .
    3. 张君君. 基于卷积神经网络的信息融合技术应用研究. 信息记录材料. 2022(07): 43-45 .

    Other cited types(3)

Catalog

    Get Citation
    PDF
    XML
    Article views (183) PDF downloads (39) Cited by(6)
    Related

    Copyright © 《Infrared Technology》Editorial Office滇ICP备12000354号-3

    Supported by: Beijing Renhe Information Technology Co., Ltd.  Baidu Analytics

    /

    DownLoad:  Full-Size Img  PowerPoint
    Return
    Return