基于全卷积网络的红外图像非均匀性校正算法

牟新刚, 崔健, 周晓

牟新刚, 崔健, 周晓. 基于全卷积网络的红外图像非均匀性校正算法[J]. 红外技术, 2022, 44(1): 21-27.
引用本文: 牟新刚, 崔健, 周晓. 基于全卷积网络的红外图像非均匀性校正算法[J]. 红外技术, 2022, 44(1): 21-27.
MOU Xingang, CUI Jian, ZHOU Xiao. Infrared Image Non-uniformity Correction Algorithm Based on Full Convolutional Network[J]. Infrared Technology , 2022, 44(1): 21-27.
Citation: MOU Xingang, CUI Jian, ZHOU Xiao. Infrared Image Non-uniformity Correction Algorithm Based on Full Convolutional Network[J]. Infrared Technology , 2022, 44(1): 21-27.

基于全卷积网络的红外图像非均匀性校正算法

基金项目: 

国家自然科学基金项目 61701357

中央高校基本科研业务费专项资金资助 183204007

详细信息
    作者简介:

    牟新刚(1982-),男,博士,副教授,主要研究方向光电成像与信息处理、红外图像处理。E-mail: sunnymou@whut.edu.cn

  • 中图分类号: TN219; TN911.73

Infrared Image Non-uniformity Correction Algorithm Based on Full Convolutional Network

  • 摘要: 针对红外成像系统在经过两点校正后,随时间漂移仍然会出现的非均匀性噪声,提出一种基于全卷积深度学习网络的红外图像非均匀性校正算法,使用子网络与主网络相结合的方式进行非均匀性校正。该算法设计了非均匀性等级估计子网络,将含有非均匀性噪声的红外图像输入子网络后,输出非均匀性等级估计图,并和待校正红外图像一并输入校正主网络。子网络生成的非均匀性等级估计图作为一个参数输入校正主网络,避免了网络只针对同一等级非均匀性产生过拟合。经过实验验证,该算法克服了传统的基于场景的算法所产生的边缘模糊问题,对条纹状非均匀性噪声校正效果较好,经过校正后的红外图像清晰度高、细节丰富、边缘清晰、图像质量良好。
    Abstract: The infrared imaging system will still exhibit non-uniform noise after two-point correction. An infrared image non-uniformity correction algorithm based on a fully convolutional deep learning network was proposed in response to this problem. This algorithm combines the subnetwork and main network for non-uniformity correction. The network contains a nonuniformity-level estimation subnetwork. After inputting the infrared image with non-uniform noise into the non-uniformity level estimation subnetwork, the outputted non-uniformity level estimation map is input into the main network together with the original noise infrared image. The non-uniformity level estimated map generated by the subnetwork prevents the network from overfitting only for the non-uniformity of the same grade. After experimental verification, the algorithm overcomes the problem of edge blur generated by the scene-based algorithm. The algorithm will not appear blurred, the images have high definition and rich details, and the quality of images is good.
  • 图  1   基于全卷积网络的特征算法实施流程

    Figure  1.   Algorithm of features based on fully convolutional network flow chart

    图  2   全卷积神经网络结构

    Figure  2.   Structure of the fully convolutional network

    图  3   非均匀性等级估计子网络

    Figure  3.   Non-uniformity level estimation subnetwork

    图  4   校正主网络

    Figure  4.   Main network of correction

    图  5   测试集PSNR图

    Figure  5.   PSNR of test data set

    图  6   测试集SSIM图

    Figure  6.   SSIM of test data set

    图  7   各算法校正效果

    Figure  7.   Results of each algorithm

    图  8   原始图片

    Figure  8.   Original Picture

    图  9   各算法校正效果

    Figure  9.   Results of eachcorrection

    表  1   子网络参数

    Table  1   Parameters of the subnetwork

    Layer Filters Input Output
    Conv 3×3×32 256×256×1 256×256×32
    Conv 3×3×32 256×256×32 256×256×32
    Conv 3×3×32 256×256×32 256×256×32
    Conv 3×3×1 256×256×32 256×256×1
    下载: 导出CSV

    表  2   校正主网络参数

    Table  2   Parameters of main network

    Layer Filters Input Output
    Conv 3×3×64 256×256×64 256×256×64
    Conv 3×3×64 256×256×64 256×256×64
    Conv 3×3×64 256×256×64 256×256×64
    Conv 3×3×64 256×256×64 256×256×64
    Conv 3×3×64 256×256×64 256×256×64
    Conv 3×3×64 256×256×64 256×256×64
    Conv 3×3×64 256×256×64 256×256×64
    Conv 3×3×64 256×256×64 256×256×64
    Conv 3×3×64 256×256×64 256×256×64
    Conv 3×3×64 256×256×64 256×256×64
    Conv 3×3×64 256×256×64 256×256×64
    Conv 3×3×1 256×256×1 256×256×1
    下载: 导出CSV

    表  3   各算法平均PSNR和SSIM

    Table  3   PSNR and SSIM of each algorithm

    Algorithm BFTH MHE DLS DMRN Ours
    PSNR/dB 32.93 33.92 34.39 34.86 35.90
    SSIM 0.828 0.858 0.887 0.903 0.928
    下载: 导出CSV

    表  4   各算法平均粗糙度

    Table  4   Roughness of each algorithm

    Algorithm BFTH MHE DLS DMRN Ours
    ρ 0.107 0.072 0.064 0.058 0.049
    下载: 导出CSV
  • [1]

    ZHOU Huixin, LI Qing, LIU Shangqian, et al. Nonuniformity and its correction principle of infrared focal plane arrays[J]. Laser & Infrared, 2003, 3(6): 46-48. http://www.researchgate.net/publication/293264376_Nonuniformity_and_its_correction_principle_of_infrared_focal_plane_arrays

    [2]

    Scribner D A, Sarkady K A, Kruer M R, et al. Adaptive nonuniformity correction for IR focal-plane arrays using neural networks[C]// International Society for Optics and Photonics, 1991: 100-109.

    [3]

    ZUO C, CHEN Q, GU G, et al. New temporal high-pass filter nonuniformity correction based on bilateral filter[J]. Optical Review, 2011, 18(2): 197-202. DOI: 10.1007/s10043-011-0042-y

    [4]

    QIAN W, CHEN Q, GU G. Space low-pass and temporal high-pass nonuniformity correction algorithm[J]. Optical Review, 2010, 17(1): 24-29. DOI: 10.1007/s10043-010-0005-8

    [5]

    Harris J G, Chiang Y M. Nonuniformity correction using the constant-statistics constraint: analog and digital implementations[C]// Proceedings of SPIE - The International Society for Optical Engineering, 1997, 3061: 895-905.

    [6]

    KUANG Xiaodong, SUI Xiubao, CHEN Qian, et al. Single infrared image stripe noise removal using deep convolutional networks[J]. IEEE Photonics Journal, 2017, 9(4): 1-13. http://www.onacademic.com/detail/journal_1000039958065210_b903.html

    [7] 赵春晖, 刘振龙. 改进的红外图像神经网络非均匀性校正算法[J]. 红外与激光工程, 2013, 42(4): 1079-1083. DOI: 10.3969/j.issn.1007-2276.2013.04.044

    ZHAO Chunhui, LIU Zhenlong. Improved infrared image neural network non-uniformity correction algorithm[J]. Infrared and Laser Engineering, 2013, 42(4): 1079-1083. DOI: 10.3969/j.issn.1007-2276.2013.04.044

    [8] 张龙, 董峰, 傅雨田. 基于神经网络的红外图像非均匀性校正[J]. 红外技术, 2018, 40(2): 164-169. http://hwjs.nvir.cn/article/id/hwjs201802011

    ZHANG Long, DONG Feng, FU Yutian. Non-uniformity correction for infrared image using neural networks[J]. Infrared Technology, 2018, 40(2): 164-169. http://hwjs.nvir.cn/article/id/hwjs201802011

    [9]

    MOU Xingang, LU Junjie, ZHOU Xiao, et al. Single frame infrared image adaptive correction algorithm based on residual network[C]//The 11th International Symposium on Photonics and Optoelectronics(SOPO), 2018: 17-23.

    [10] 牟新刚, 陆俊杰, 周晓. 基于残差编解码网络的红外图像自适应校正算法[J]. 红外技术, 2020, 42(9): 833-839. http://hwjs.nvir.cn/article/id/hwjs202009004

    MOU Xingang, LU Junjie, ZHOU Xiao. Adaptive correction algorithm of infrared image based on encoding and decoding residual network[J]. Infrared Technology, 2020, 42(9): 833-839. http://hwjs.nvir.cn/article/id/hwjs202009004

    [11]

    HE Zewei, CAO Yanpeng, DONG Yafei, et al. Single-image-based nonuniformity correction of uncooled long-wave infrared detectors: a deep-learning approach[J]. Applied Optics, 2018, 57(18): 155-164 DOI: 10.1364/AO.57.00D155

    [12]

    ZHANG Kai, ZUO Wangmeng, ZHANG Lei. Ffdnet: Toward a fast and flexible solution for CNN based image denoising[C]//IEEE Transactions on Image Processing, 2017: 4608-4622.

    [13]

    GUO Shi, YAN Zifei, ZHANG Kai, et al. Toward convolutional blind denoising of real photographs[C]//IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2019: 1712-1722.

    [14]

    Ioffe S, Szegedy C. Batch normalization: Accelerating deep network training by reducing internal covariate shift[C]//International Conference on Machine Learning, 2015: 448-456.

    [15]

    QIAN W, CHEN Q, GU G. Space low-pass and temporal high-pass nonuniformity correction algorithm[J]. Optical Review, 2010, 17(1): 24-29. DOI: 10.1007/s10043-010-0005-8

    [16]

    CHANG Y, YAN L, LIU L, et al. Infrared Aerothermal nonuniform correction via deep multiscale residual network[J]. IEEE Geoscience and Remote Sensing Letters, 2019, 16(7): 1120-1124. DOI: 10.1109/LGRS.2019.2893519

  • 期刊类型引用(8)

    1. 郭广浩,吴南健,刘力源. 低功耗非制冷红外图像处理专用SoC芯片. 红外与毫米波学报. 2023(01): 122-131 . 百度学术
    2. 郭玉婷,贾晓洪,李丽娟,刘俊明. 时空域自适应滤波非均匀性校正算法. 红外技术. 2023(05): 482-487 . 本站查看
    3. 黎明清 ,王宇庆 ,孙海江 . 基于侧窗滤波改进的神经网络非均匀性校正算法. 液晶与显示. 2023(11): 1567-1579 . 百度学术
    4. 汪晓,葛军. 基于局部梯度的神经网络非均匀性校正算法. 红外. 2018(03): 18-22+43 . 百度学术
    5. 潘科辰,顾国华,陈钱,隋修宝. 一种改进的全变分模型校正红外焦平面阵列条纹非均匀性. 红外技术. 2016(02): 138-143 . 本站查看
    6. 李召龙,沈同圣,史浩然,娄树理. 红外成像系统非均匀性快速校正方法. 舰船科学技术. 2016(21): 148-150 . 百度学术
    7. 牛英宇,聂瑞杰,李丽娟. 基于FPGA的红外图像非均匀校正实现方法. 激光与红外. 2016(08): 1028-1032 . 百度学术
    8. 陈芳林,张宝辉,汪贵华,杨开峰,于世孔. 改进的神经网络非均匀性校正算法研究. 科学技术与工程. 2016(33): 215-220 . 百度学术

    其他类型引用(8)

图(9)  /  表(4)
计量
  • 文章访问数:  335
  • HTML全文浏览量:  97
  • PDF下载量:  98
  • 被引次数: 16
出版历程
  • 收稿日期:  2021-02-07
  • 修回日期:  2021-04-24
  • 刊出日期:  2022-01-19

目录

    /

    返回文章
    返回