[1]王 丹,陈 亮.基于深度学习的红外夜视图像超分辨率重建[J].红外技术,2019,41(10):963-969.[doi:doi:10.11846/j.issn.1001_8891.201910012]
 WANG Dan,CHEN Liang.Super-resolution Reconstruction of Infrared Images in Night Environments Based on Deep-learning [J].Infrared Technology,2019,41(10):963-969.[doi:doi:10.11846/j.issn.1001_8891.201910012]
点击复制

基于深度学习的红外夜视图像超分辨率重建
分享到:

《红外技术》[ISSN:1001-8891/CN:CN 53-1053/TN]

卷:
41卷
期数:
2019年第10期
页码:
963-969
栏目:
出版日期:
2019-10-21

文章信息/Info

Title:
Super-resolution Reconstruction of Infrared Images in Night Environments
Based on Deep-learning
文章编号:
1001-8891(2019)10-0963-07
作者:
王 丹陈 亮
东华大学 信息科学与技术学院,上海 201600
Author(s):
WANG DanCHEN Liang
School of Information Science and Technology, DongHua University, Shanghai 201600, China
关键词:
红外夜视图像超分辨率预处理超深神经网络
Keywords:
infrared image in night environment super-resolution preprocessing very deep neural network
分类号:
TP183
DOI:
doi:10.11846/j.issn.1001_8891.201910012
文献标志码:
A
摘要:
针对红外夜视图像对比度低、成像质量不高的问题,提出适合红外夜视图像超分辨率重建方法。在自然图像超分辨率重建模型的基础上增加基于Retinex的对比度增强预处理步骤,并对网络模型做如下改进:构建超深卷积神经网络学习低分辨率图像与高分辨率图像之间的映射关系,增大感受野,提升网络学习能力;仅学习高低分辨率图像间的差值信息加速网络收敛。针对高分辨率红外夜视图像不易获得,数据量较少的问题,利用迁移学习理论,使用少量的高分辨率红外夜视图像为目标样本,对自然图像超分辨率重建模型进行微调,得到适合红外夜视图像重建的网络权重模型。实验结果证明:使用该方法得到的红外夜视图像信息丰富,层次分明,具有良好的视觉效果。
Abstract:
To address the low contrast and poor image quality of infrared images recorded in a night environment(NIR), we propose an algorithm suitable for NIR super-resolution. Based on a color image super-resolution reconstruction model, a preprocessing step based on Retinex is added to enhance the contrast and improve the convolution neural network as follows: First, more layers are used to increase the receptive field to improve the learning ability of the network. Second, the residual image generated by low- and high-resolution images is learned to improve the convergence speed. Finally, the network trained by high-resolution images is fine-tuned to obtain weights that fit the NIR reconstruction with a small amount of data according to transfer learning, because the lack of data makes it difficult to get more NIR images. Simulation results shows that the proposed algorithm performs well, yielding a reconstructed image with higher contrast, richer detail, and better visuals.

参考文献/References:

[1] Gyaourova A, Bebis G, Pavlidis I. Fusion of infrared and visible images for face recognition computer vision[C]//ECCV 2004, Springer Berlin Heidelberg, 2004: 456-468.
[2] MA J , CHEN C , LI C , et al. Infrared and visible image fusion via gradient transfer and total variation minimization[J]. Information Fusion, 2016, 31:100-109.
[3] Bavirisetti D P, Dhuli R. Fusion of infrared and visible sensor images based on anisotropic diffusion and Karhunen-loeve transform[J]. IEEE Sensors Journal, 2015, 16(1): 203-209.
[4] ZHAO Y, CHEN Q, SUI X, et al. A novel infrared image super-resolution method based on sparse representation[J]. Infrared Physics & Technology, 2015, 71: 506-513.
[5] DONG C, CHEN C L, HE K, et al. Learning a deep convolutional network for image super -resolution[C]//Proceedings of the 2014 European Conference on Computer Vision, 2014, 8692: 184-199.
[6] YANG J, Wright J, HUANG T, et al. Image super-resolution as sparse representation of raw image patches[C]// Computer Vision and Pattern Recognition, 2008 CVPR of IEEE Conference, 2008: 1-8.
[7] DONG C, CHEN C L, TANG X. Accelerating the super-resolution convolutional neural network[C]//Proceedings of the 2016 European Conference on Computer Vision, 2016: 391-407.
[8] Ma Wenye, Osher S. A TV Bregman iterative model of Retinex theory[J]. Inverse Problems & Imaging, 2012, 6(4): 697-708.
[9] SHI W, Caballero J, Huszár F, et al. Real-time single image and video super-resolution using an efficient sub-pixel convolutional neural network[C]// IEEE Conference on Computer Vision and Pattern Recognition, 2016: 1874-1883.
[10] Ganin Y , Ustinova E , Ajakan H , et al. Domain-adversarial training of neural networks[J]. Journal of Machine Learning Research, 2015, 17(1): 2096-2030.
[11] Yosinski J, Clune J, Bengio Y, et al. How transferable are features in deep neural networks[J]. Eprint Arxiv, 2014, 27: 3320-3328.
[12] Mittal A, Moorthy A K, Bovik A C. Blind/referenceless image spatial quality evaluator[C]// Signals, Systems and Computers of IEEE Conference, 2012: 723-727.

相似文献/References:

[1]聂笃宪,陈一梅,陈鹤峰.去混合噪声的超分辨率图像重建算法[J].红外技术,2010,32(10):604.
 NIE Du-xian,CHEN Yi-mei,CHEN He-feng.Super-resolution Image Reconstruction Algorithm for mixed Noises Removal[J].Infrared Technology,2010,32(10):604.
[2]樊超,孙宁宁,夏旭.基于序列图像的超分辨率重建[J].红外技术,2010,32(5):279.
 FAN Chao,SUN Ning-ning,XIA Xu.Super-resolution Reconstruction Based on Image Sequences[J].Infrared Technology,2010,32(10):279.
[3]陈博洋,陈桂林,孙胜利.亚像元技术在图像采集系统中的应用[J].红外技术,2007,29(4):226.
 CHEN Bo-Yang,CHEN Gui-Lin,SUN Sheng-Li.Application of Subpixel Technology for Image Collection System[J].Infrared Technology,2007,29(10):226.
[4]浦利,金伟其,刘玉树,等.基于小波双立方配比插值的图像插值放大算法研究[J].红外技术,2006,28(8):453.
 PU Li,JIN Wei-qi,LIU Yu-shu,et al.A Study of Wavelet Bi-cubic Ratio Interpolation Algorithm[J].Infrared Technology,2006,28(10):453.
[5]徐宏财,向健勇,潘皓.一种改进的POCS算法的超分辨率图像重建[J].红外技术,2005,27(6):477.
 XU Hong-cai,XIANG Jian-yong,PAN Hao.An Improved POCS Algorithm for Super-resolution Image Reconstruction[J].Infrared Technology,2005,27(10):477.
[6]姚 敏,周 勤.一种低匹配误差敏感度的红外图像超分辨率算法[J].红外技术,2016,38(10):864.[doi:10.11846/j.issn.1001_8891.201610010]
 YAO Min,ZHOU Qin.An Infrared Images Super-resolution Algorithm with Low Registration Error Sensitivity [J].Infrared Technology,2016,38(10):864.[doi:10.11846/j.issn.1001_8891.201610010]
[7]刘 哲,黄世奇,姜 杰.基于引导滤波和多尺度局部自相似单幅红外图像超分辨率方法[J].红外技术,2017,39(4):345.[doi:10.11846/j.issn.1001_8891.201704009]
 LIU Zhe,HUANG Shiqi,JIANG Jie.Single Image Super Resolution Method Based on Multi-scale Self-similarity and Non Local Means [J].Infrared Technology,2017,39(10):345.[doi:10.11846/j.issn.1001_8891.201704009]
[8]刘 哲,韩九强,黄世奇.基于多引导滤波器的单幅图像超分辨率技术[J].红外技术,2017,39(10):920.[doi:10.11846/j.issn.1001_8891.201710009]
 LIU Zhe,HAN jiuqiang,HUANG Shiqi.Single Image Super-Resolution Based on Multi-Guided Filtering[J].Infrared Technology,2017,39(10):920.[doi:10.11846/j.issn.1001_8891.201710009]
[9]姜杰,刘哲,吕林涛.局部线性嵌入的快速单幅图像超分辨率技术[J].红外技术,2018,40(1):039.[doi:10.11846/j.issn.1001_8891.201801008]
 JIANG Jie,LIU Zhe,LV Lintao.Fast Single-image Super Resolution Technique Based on Local Linear Embedding[J].Infrared Technology,2018,40(10):039.[doi:10.11846/j.issn.1001_8891.201801008]
[10]蔡坤琪.快速图像超分辨率方法研究[J].红外技术,2018,40(3):269.[doi:10.11846/j.issn.1001_8891.201803012]
 CAI Kunqi.A Study on Rapid Image Super-resolution[J].Infrared Technology,2018,40(10):269.[doi:10.11846/j.issn.1001_8891.201803012]

备注/Memo

备注/Memo:
收稿日期:2018-09-01;修订日期:2019-03-31.
作者简介:王丹(1993-),女,硕士研究生,主要研究方向:深度学习、图像超分辨率。
通信作者:陈亮(1976-),女,博士,副教授,混沌系统动力学分析与控制,系统建模与仿真,神经网络及其机器学习等。E-mail:chenliang@dhu.edu.cn。
更新日期/Last Update: 2019-10-23