留言板

尊敬的读者、作者、审稿人, 关于本刊的投稿、审稿、编辑和出版的任何问题, 您可以本页添加留言。我们将尽快给您答复。谢谢您的支持!

姓名
邮箱
手机号码
标题
留言内容
验证码

基于深度卷积神经网络的红外图像超分辨率重建技术

袁茜琳 张宝辉 张倩 何铭 周金杰 练琤 岳江

袁茜琳, 张宝辉, 张倩, 何铭, 周金杰, 练琤, 岳江. 基于深度卷积神经网络的红外图像超分辨率重建技术[J]. 红外技术, 2023, 45(5): 498-505.
引用本文: 袁茜琳, 张宝辉, 张倩, 何铭, 周金杰, 练琤, 岳江. 基于深度卷积神经网络的红外图像超分辨率重建技术[J]. 红外技术, 2023, 45(5): 498-505.
YUAN Xilin, ZHANG Baohui, ZHANG Qian, HE Ming, ZHOU Jinjie, LIAN Cheng, YUE Jiang. Infrared Images with Super-resolution Based on Deep Convolutional Neural Network[J]. Infrared Technology , 2023, 45(5): 498-505.
Citation: YUAN Xilin, ZHANG Baohui, ZHANG Qian, HE Ming, ZHOU Jinjie, LIAN Cheng, YUE Jiang. Infrared Images with Super-resolution Based on Deep Convolutional Neural Network[J]. Infrared Technology , 2023, 45(5): 498-505.

基于深度卷积神经网络的红外图像超分辨率重建技术

详细信息
    作者简介:

    袁茜琳(2000-),女,硕士研究生,主要从事红外图像处理技术方面的研究。E-mail:yuanxilinnn@163.com

    通讯作者:

    张宝辉(1984-),男,正高级工程师,博士,主要从事红外探测与图像处理方面的研究。E-mail:zbhmatt@163.com

  • 中图分类号: TN219

Infrared Images with Super-resolution Based on Deep Convolutional Neural Network

  • 摘要: 由于器件及工艺等技术限制,红外图像分辨率相对可见光图像较低,存在细节纹理特征模糊等不足。对此,本文提出一种基于深度卷积神经网络(convolutional neural network,CNN)的红外图像超分辨率重建方法。该方法改进残差模块,降低激活函数对信息流影响的同时加深网络,充分利用低分辨率红外图像的原始信息。结合高效通道注意力机制和通道-空间注意力模块,使重建过程中有选择性地捕获更多特征信息,有利于对红外图像高频细节更准确地进行重建。实验结果表明,本文方法重建红外图像峰值信噪比(peak signal to noise ratio,PSNR)优于传统的Bicubic插值法以及基于CNN的SRResNet、EDSR、RCAN模型。当尺度因子为×2和×4时,重建图像的平均PSNR值比传统Bicubic插值法分别提高了4.57 dB和3.37 dB。
  • 图  1  基于CNN的高效残差注意力网络结构(ERAN)

    Figure  1.  Efficient residual attention network (ERAN) based on CNN

    图  2  高效残差注意力块(ERAB)结构

    Figure  2.  Structure of efficient residual attention block (ERAB)

    图  3  不同残差模块结构对比

    Figure  3.  Comparison of different residual module structures

    图  4  “TEST_1”使用不同超分辨率重建方法的重建结果(尺度因子为×2)

    Figure  4.  Reconstructed results of "TEST_1" by different super-resolution reconstruction approaches(upscaling factor is ×2)

    图  5  “TEST_2”使用不同超分辨率重建方法的重建结果(尺度因子为×2)

    Figure  5.  Reconstructed results of "TEST_2" by different super-resolution reconstruction approaches(upscaling factor is ×2)

    图  6  “TEST_3”使用不同超分辨率重建方法的重建结果(尺度因子为×4)

    Figure  6.  Reconstructed results of "TEST_3" by different super-resolution reconstruction approaches(upscaling factor is ×4)

    图  7  “TEST_4”使用不同超分辨率重建方法的重建结果(尺度因子为×4)

    Figure  7.  Reconstructed result of "TEST_4" by different super-resolution reconstruction approaches(upscaling factor is ×4)

    表  1  ERAB(包括IRB和ECA)和CSA的研究

    Table  1.   Investigations of ERAB (including IRB and ECA) and CSA

    Base M1 M2 M3 M4
    IRB
    CSA
    ECA
    PSNR/dB 42.44 42.78 42.97 42.86 43.06
    SSIM 0.9543 0.9549 0.9552 0.9550 0.9554
    下载: 导出CSV

    表  2  典型图像实验结果对比

    Table  2.   Comparison of typical images experiment results

    Images Scale Bicubic
    (PSNR/dB)/SSIM
    SRResNet
    (PSNR/dB)/SSIM
    EDSR
    (PSNR/dB)/SSIM
    RCAN
    (PSNR/dB)/SSIM
    Ours
    (PSNR/dB)/SSIM
    TEST_1 ×2 38.0631/0.9710 43.0650/0.9868 43.1183/0.9869 43.1627/0.9870 43.3488/0.9875
    TEST_2 30.8528/0.7505 32.1850/0.7875 32.1907/0.7878 32.1908/0.7880 32.2152/0.7881
    TEST_3 ×4 31.8568/0.9091 35.4462/0.9498 35.3748/0.9493 35.7971/0.9516 36.0539/0.9538
    TEST_4 28.6532/0.8888 35.3544/0.9551 35.3770/0.9548 35.5273/0.9559 36.1061/0.9586
    下载: 导出CSV

    表  3  测试图像集超分辨率重建结果比较

    Table  3.   Comparison of super-resolution reconstruction results of test image set

    Methods (PSNR/dB)/SSIM(×2) (PSNR/dB)/SSIM(×4)
    Bicubic 38.4937/0.9395 32.3882/0.8699
    SRResNet 41.6272/0.9220 34.8284/0.8999
    EDSR 41.6705/0.9221 35.4601/0.9035
    RCAN 42.5441/0.9546 35.5871/0.9042
    OURS 43.0616/0.9554 35.7599/0.9053
    下载: 导出CSV
  • [1] Baker S, Kanade T. Limits on super resolution and how to break them[J]. IEEE Transactions on Pattern Analysis & Machine Intelligence, 2002, 24(9): 1167-1183. http://www.onacademic.com/detail/journal_1000035551419310_c5fb.html
    [2] Hou H, Andrews H. Cubic splines for image interpolation and digital filtering[J]. IEEE Transactions on acoustics, speech, and signal processing, 1978, 26(6): 508-517. doi:  10.1109/TASSP.1978.1163154
    [3] Freeman W T, Pasztor E C, Carmichael O T. Learning low-level vision[J]. International Journal of Computer Vision, 2000, 40(1): 25-47. doi:  10.1023/A:1026501619075
    [4] YANG J, Wright J, HUANG T S, et al. Image super-resolution via sparse representation[J]. IEEE transactions on image processing, 2010, 19(11): 2861-2873. doi:  10.1109/TIP.2010.2050625
    [5] DONG C, Loy C C, HE K, et al. Image super-resolution using deep convolutional networks[J]. IEEE Trans Pattern Anal Mach Intell, 2016, 38(2): 295-307. doi:  10.1109/TPAMI.2015.2439281
    [6] DONG C, Loy C C, TANG X. Accelerating the super-resolution convolutional neural network[C]//Proceedings of the European conference on computer vision (ECCV), 2016: 391-407.
    [7] SHI W, Caballero J, Huszár F, et al. Real-time single image and video super-resolution using an efficient sub-pixel convolutional neural network[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2016: 1874-1883.
    [8] HE K, ZHANG X, REN S, et al. Deep residual learning for image recognition[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2016: 770-778.
    [9] Ledig C, Theis L, Huszár F, et al. Photo-realistic single image super-resolution using a generative adversarial network[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2017: 4681-4690.
    [10] Lim B, Son S, Kim H, et al. Enhanced deep residual networks for single image super-resolution[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, 2017: 136-144.
    [11] ZHANG Y, LI K, LI K, et al. Image super-resolution using very deep residual channel attention networks[C]//Proceedings of the European Conference on Computer Vision (ECCV), 2018: 286-301.
    [12] DAI T, CAI J, ZHANG Y, et al. Second-order attention network for single image super-resolution[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2019: 11065-11074.
    [13] NIU B, WEN W, REN W, et al. Single image super-resolution via a holistic attention network[C]//Proceedings of the European Conference on Computer Vision (ECCV), 2020: 191-207.
    [14] Choi Y, Kim N, Hwang S, et al. Thermal image enhancement using convolutional neural network[C]//2016 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). 2016: 223-230.
    [15] HE Z, TANG S, YANG J, et al. Cascaded deep networks with multiple receptive fields for infrared image super-resolution[J]. IEEE Transactions on Circuits and Systems for Video Technology, 2018, 29(8): 2310-2322. http://www.xueshufan.com/publication/2886209123
    [16] ZOU Y, ZHANG L, LIU C, et al. Super-resolution reconstruction of infrared images based on a convolutional neural network with skip connections[J]. Optics and Lasers in Engineering, 2021, 146: 106717. doi:  10.1016/j.optlaseng.2021.106717
    [17] YU J, FAN Y, YANG J, et al. Wide activation for efficient and accurate image super-resolution[J/OL]. arXiv preprint arXiv: 1808.08718, 2018.
    [18] WANG Q, WU B, ZHU P, et al. ECA-Net: Efficient channel attention for deep convolutional neural networks[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2020: 11534-11542.
  • 加载中
图(7) / 表(3)
计量
  • 文章访问数:  154
  • HTML全文浏览量:  48
  • PDF下载量:  53
  • 被引次数: 0
出版历程
  • 收稿日期:  2022-12-30
  • 修回日期:  2023-02-20
  • 刊出日期:  2023-05-20

目录

    /

    返回文章
    返回