基于全卷积网络的红外图像非均匀性校正算法

牟新刚; 崔健; 周晓

基于全卷积网络的红外图像非均匀性校正算法

武汉理工大学机电工程学院，湖北武汉 430070

基金项目:

国家自然科学基金项目 61701357

中央高校基本科研业务费专项资金资助 183204007

详细信息

作者简介:
牟新刚（1982-），男，博士，副教授，主要研究方向光电成像与信息处理、红外图像处理。E-mail: sunnymou@whut.edu.cn

中图分类号: TN219; TN911.73
计量
- 文章访问数: 259
- HTML全文浏览量: 77
- PDF下载量: 79
- 被引次数: 0
出版历程
- 收稿日期: 2021-02-08
- 修回日期: 2021-04-25
- 刊出日期: 2022-01-20

Infrared Image Non-uniformity Correction Algorithm Based on Full Convolutional Network

School of Mechanical and Electronic Engineering, Wuhan University of Technology, Wuhan 430070, China

摘要

摘要: 针对红外成像系统在经过两点校正后，随时间漂移仍然会出现的非均匀性噪声，提出一种基于全卷积深度学习网络的红外图像非均匀性校正算法，使用子网络与主网络相结合的方式进行非均匀性校正。该算法设计了非均匀性等级估计子网络，将含有非均匀性噪声的红外图像输入子网络后，输出非均匀性等级估计图，并和待校正红外图像一并输入校正主网络。子网络生成的非均匀性等级估计图作为一个参数输入校正主网络，避免了网络只针对同一等级非均匀性产生过拟合。经过实验验证，该算法克服了传统的基于场景的算法所产生的边缘模糊问题，对条纹状非均匀性噪声校正效果较好，经过校正后的红外图像清晰度高、细节丰富、边缘清晰、图像质量良好。
- 非均匀性等级估计子网络 /
- 红外图像 /
- 非均匀性校正 /
- 深度学习
Abstract: The infrared imaging system will still exhibit non-uniform noise after two-point correction. An infrared image non-uniformity correction algorithm based on a fully convolutional deep learning network was proposed in response to this problem. This algorithm combines the subnetwork and main network for non-uniformity correction. The network contains a nonuniformity-level estimation subnetwork. After inputting the infrared image with non-uniform noise into the non-uniformity level estimation subnetwork, the outputted non-uniformity level estimation map is input into the main network together with the original noise infrared image. The non-uniformity level estimated map generated by the subnetwork prevents the network from overfitting only for the non-uniformity of the same grade. After experimental verification, the algorithm overcomes the problem of edge blur generated by the scene-based algorithm. The algorithm will not appear blurred, the images have high definition and rich details, and the quality of images is good.
- non-uniformity level estimation subnetwork /
- infrared image /
- non-uniformity correction /
- deep learning

HTML全文

图 1 基于全卷积网络的特征算法实施流程

Figure 1. Algorithm of features based on fully convolutional network flow chart

下载: 全尺寸图片幻灯片

图 2 全卷积神经网络结构

Figure 2. Structure of the fully convolutional network

下载: 全尺寸图片幻灯片

图 3 非均匀性等级估计子网络

Figure 3. Non-uniformity level estimation subnetwork

下载: 全尺寸图片幻灯片

图 4 校正主网络

Figure 4. Main network of correction

下载: 全尺寸图片幻灯片

图 5 测试集PSNR图

Figure 5. PSNR of test data set

下载: 全尺寸图片幻灯片

图 6 测试集SSIM图

Figure 6. SSIM of test data set

下载: 全尺寸图片幻灯片

图 7 各算法校正效果

Figure 7. Results of each algorithm

下载: 全尺寸图片幻灯片

图 8 原始图片

Figure 8. Original Picture

下载: 全尺寸图片幻灯片

图 9 各算法校正效果

Figure 9. Results of eachcorrection

下载: 全尺寸图片幻灯片

表 1 子网络参数

Table 1. Parameters of the subnetwork

Layer	Filters	Input	Output
Conv	3×3×32	256×256×1	256×256×32
Conv	3×3×32	256×256×32	256×256×32
Conv	3×3×32	256×256×32	256×256×32
Conv	3×3×1	256×256×32	256×256×1

下载: 导出CSV

表 2 校正主网络参数

Table 2. Parameters of main network

Layer	Filters	Input	Output
Conv	3×3×64	256×256×64	256×256×64
Conv	3×3×64	256×256×64	256×256×64
Conv	3×3×64	256×256×64	256×256×64
Conv	3×3×64	256×256×64	256×256×64
Conv	3×3×64	256×256×64	256×256×64
Conv	3×3×64	256×256×64	256×256×64
Conv	3×3×64	256×256×64	256×256×64
Conv	3×3×64	256×256×64	256×256×64
Conv	3×3×64	256×256×64	256×256×64
Conv	3×3×64	256×256×64	256×256×64
Conv	3×3×64	256×256×64	256×256×64
Conv	3×3×1	256×256×1	256×256×1

下载: 导出CSV

表 3 各算法平均PSNR和SSIM

Table 3. PSNR and SSIM of each algorithm

Algorithm	BFTH	MHE	DLS	DMRN	Ours
PSNR/dB	32.93	33.92	34.39	34.86	35.90
SSIM	0.828	0.858	0.887	0.903	0.928

下载: 导出CSV

表 4 各算法平均粗糙度

Table 4. Roughness of each algorithm

Algorithm	BFTH	MHE	DLS	DMRN	Ours
ρ	0.107	0.072	0.064	0.058	0.049

下载: 导出CSV

参考文献(16)

[1]	ZHOU Huixin, LI Qing, LIU Shangqian, et al. Nonuniformity and its correction principle of infrared focal plane arrays[J]. Laser & Infrared, 2003, 3(6): 46-48. http://www.researchgate.net/publication/293264376_Nonuniformity_and_its_correction_principle_of_infrared_focal_plane_arrays
[2]	Scribner D A, Sarkady K A, Kruer M R, et al. Adaptive nonuniformity correction for IR focal-plane arrays using neural networks[C]// International Society for Optics and Photonics, 1991: 100-109.
[3]	ZUO C, CHEN Q, GU G, et al. New temporal high-pass filter nonuniformity correction based on bilateral filter[J]. Optical Review, 2011, 18(2): 197-202. doi: 10.1007/s10043-011-0042-y
[4]	QIAN W, CHEN Q, GU G. Space low-pass and temporal high-pass nonuniformity correction algorithm[J]. Optical Review, 2010, 17(1): 24-29. doi: 10.1007/s10043-010-0005-8
[5]	Harris J G, Chiang Y M. Nonuniformity correction using the constant-statistics constraint: analog and digital implementations[C]// Proceedings of SPIE - The International Society for Optical Engineering, 1997, 3061: 895-905.
[6]	KUANG Xiaodong, SUI Xiubao, CHEN Qian, et al. Single infrared image stripe noise removal using deep convolutional networks[J]. IEEE Photonics Journal, 2017, 9(4): 1-13. http://www.onacademic.com/detail/journal_1000039958065210_b903.html
[7]	赵春晖, 刘振龙. 改进的红外图像神经网络非均匀性校正算法[J]. 红外与激光工程, 2013, 42(4): 1079-1083. doi: 10.3969/j.issn.1007-2276.2013.04.044 ZHAO Chunhui, LIU Zhenlong. Improved infrared image neural network non-uniformity correction algorithm[J]. Infrared and Laser Engineering, 2013, 42(4): 1079-1083. doi: 10.3969/j.issn.1007-2276.2013.04.044
[8]	张龙, 董峰, 傅雨田. 基于神经网络的红外图像非均匀性校正[J]. 红外技术, 2018, 40(2): 164-169. http://hwjs.nvir.cn/article/id/hwjs201802011 ZHANG Long, DONG Feng, FU Yutian. Non-uniformity correction for infrared image using neural networks[J]. Infrared Technology, 2018, 40(2): 164-169. http://hwjs.nvir.cn/article/id/hwjs201802011
[9]	MOU Xingang, LU Junjie, ZHOU Xiao, et al. Single frame infrared image adaptive correction algorithm based on residual network[C]//The 11th International Symposium on Photonics and Optoelectronics(SOPO), 2018: 17-23.
[10]	牟新刚, 陆俊杰, 周晓. 基于残差编解码网络的红外图像自适应校正算法[J]. 红外技术, 2020, 42(9): 833-839. http://hwjs.nvir.cn/article/id/hwjs202009004 MOU Xingang, LU Junjie, ZHOU Xiao. Adaptive correction algorithm of infrared image based on encoding and decoding residual network[J]. Infrared Technology, 2020, 42(9): 833-839. http://hwjs.nvir.cn/article/id/hwjs202009004
[11]	HE Zewei, CAO Yanpeng, DONG Yafei, et al. Single-image-based nonuniformity correction of uncooled long-wave infrared detectors: a deep-learning approach[J]. Applied Optics, 2018, 57(18): 155-164 doi: 10.1364/AO.57.00D155
[12]	ZHANG Kai, ZUO Wangmeng, ZHANG Lei. Ffdnet: Toward a fast and flexible solution for CNN based image denoising[C]//IEEE Transactions on Image Processing, 2017: 4608-4622.
[13]	GUO Shi, YAN Zifei, ZHANG Kai, et al. Toward convolutional blind denoising of real photographs[C]//IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2019: 1712-1722.
[14]	Ioffe S, Szegedy C. Batch normalization: Accelerating deep network training by reducing internal covariate shift[C]//International Conference on Machine Learning, 2015: 448-456.
[15]	QIAN W, CHEN Q, GU G. Space low-pass and temporal high-pass nonuniformity correction algorithm[J]. Optical Review, 2010, 17(1): 24-29. doi: 10.1007/s10043-010-0005-8
[16]	CHANG Y, YAN L, LIU L, et al. Infrared Aerothermal nonuniform correction via deep multiscale residual network[J]. IEEE Geoscience and Remote Sensing Letters, 2019, 16(7): 1120-1124. doi: 10.1109/LGRS.2019.2893519