Single-frame Infrared Image Super-Resolution Reconstruction for Real Scenes

SHI Yifeng; CHEN Nan; ZHU Fang; MAO Wenbiao; LI Faming; WANG Tianfu; ZHANG Jiqing; YAO Libin

Volume 46 Issue 4

Apr. 2024

Turn off MathJax

Article Contents

Article Navigation > Infrared Technology > 2024 > 46(4): 427-436

SHI Yifeng, CHEN Nan, ZHU Fang, MAO Wenbiao, LI Faming, WANG Tianfu, ZHANG Jiqing, YAO Libin. Single-frame Infrared Image Super-Resolution Reconstruction for Real Scenes[J]. Infrared Technology , 2024, 46(4): 427-436.

Citation:

PDF( 1863 KB)

Single-frame Infrared Image Super-Resolution Reconstruction for Real Scenes

Kunming Institute of Physics, Kunming 650223, China

Received Date: 2023-12-06
Rev Recd Date: 2024-01-19
Publish Date: 2024-04-20

Abstract

Abstract

Current infrared image super-resolution reconstruction methods, which are primarily designed based on experimental data, often fail in complex degradation scenarios encountered in real-world environments. To address this challenge, this paper presents a novel deep learning-based approach tailored for the super-resolution reconstruction of infrared images in real scenarios. The significant contributions of this research include the development of a model that simulates infrared image degradation in real-life settings and a network structure that integrates channel attention with dense connections. This structure enhances feature extraction and image reconstruction capabilities, effectively increasing the spatial resolution of low-resolution infrared images in realistic scenarios. The effectiveness and superiority of the proposed approach for processing infrared images in real-world contexts are demonstrated through a series of ablation studies and comparative experiments with existing super-resolution methods. The experimental results indicate that this method produces sharper edges and effectively eliminates noise and blur, thereby significantly improving the visual quality of the images.
- infrared image,
- deep learning,
- super-resolution,
- real scene,
- degradation model

FullText(HTML)

References(27)

References

[1]	WANG Z, CHEN J, Hoi S C H. Deep learning for image super-resolution: A survey[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2020, 43(10): 3365-3387.
[2]	LI J, PEI Z, ZENG T. From beginner to master: A survey for deep learning-based single-image super-resolution[J]. arXiv preprint arXiv: 2109.14335, 2021.
[3]	DONG C, LOY C C, HE K, et al. Image super-resolution using deep convolutional networks[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2015, 38(2): 295-307.
[4]	SHI W, Caballero J, Huszár F, et al. Real-time single image and video super-resolution using an efficient sub-pixel convolutional neural network[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2016: 1874-1883.
[5]	LIM B, SON S, KIM H, et al. Enhanced deep residual networks for single image super-resolution[C]//Proceedings of The IEEE Conference on Computer Vision And Pattern Recognition Workshops, 2017: 136-144.
[6]	WANG X, YU K, WU S, et al. Esrgan: Enhanced super-resolution generative adversarial networks[C]//Proceedings of the European Conference on Computer Vision (ECCV), 2018: 63-79.
[7]	SUN C, LV J, LI J, et al. A rapid and accurate infrared image super-resolution method based on zoom mechanism[J]. Infrared Physics & Technology, 2018, 88: 228-238.
[8]	Suryanarayana G, TU E, YANG J. Infrared super-resolution imaging using multi-scale saliency and deep wavelet residuals[J]. Infrared Physics & Technology, 2019, 97: 177-186.
[9]	YAO T, LUO Y, HU J, et al. Infrared image super-resolution via discriminative dictionary and deep residual network[J]. Infrared Physics & Technology, 2020, 107: 103314.
[10]	Oz N, Sochen N, Markovich O, et al. Rapid super resolution for infrared imagery[J]. Optics Express, 2020, 28(18): 27196-27209. doi: 10.1364/OE.389926
[11]	ZOU Y, ZHANG L, LIU C, et al. Super-resolution reconstruction of infrared images based on a convolutional neural network with skip connections[J]. Optics and Lasers in Engineering, 2021, 146: 106717. doi: 10.1016/j.optlaseng.2021.106717
[12]	李方彪, 何昕, 魏仲慧, 等. 生成式对抗神经网络的多帧红外图像超分辨率重建[J]. 红外与激光工程, 2018, 47(2): 26-33. https://www.cnki.com.cn/Article/CJFDTOTAL-HWYJ201802004.htm LI F, HE X, WEI Z, et al. Multiframe infrared image super-resolution reconstruction using generative adversarial networks[J]. Infrared and Laser Engineering, 2018, 47(2): 26-33. https://www.cnki.com.cn/Article/CJFDTOTAL-HWYJ201802004.htm
[13]	魏子康, 刘云清. 改进的RDN灰度图像超分辨率重建方法[J]. 红外与激光工程, 2020, 49(S1): 20200173. https://www.cnki.com.cn/Article/CJFDTOTAL-HWYJ2020S1022.htm WEI Z, LIU Y. Gray image super-resolution reconstruction based on improved RDN method[J]. Infrared and Laser Engineering, 2020, 49(S1): 20200173. https://www.cnki.com.cn/Article/CJFDTOTAL-HWYJ2020S1022.htm
[14]	胡蕾, 王足根, 陈田, 等. 一种改进的SRGAN红外图像超分辨率重建算法[J]. 系统仿真学报, 2021, 33(9): 2109-2118. https://www.cnki.com.cn/Article/CJFDTOTAL-XTFZ202109013.htm HU L, WANG Z, CHEN T, et al. An improved SRGAN infrared image super-resolution reconstruction algorithm[J]. Journal of System Simulation, 2021, 33(9): 2109-2118. https://www.cnki.com.cn/Article/CJFDTOTAL-XTFZ202109013.htm
[15]	邱德粉, 江俊君, 胡星宇, 等. 高分辨率可见光图像引导红外图像超分辨率的Transformer网络[J]. 中国图象图形学报, 2023, 28(1): 196-206. https://www.cnki.com.cn/Article/CJFDTOTAL-ZGTB202301012.htm QIU D, JIANG J, HU X, et al. Guided transformer for high-resolution visible image guided infrared image super-resolution[J]. Journal of Image and Graphics, 2023, 28(1): 196-206. https://www.cnki.com.cn/Article/CJFDTOTAL-ZGTB202301012.htm
[16]	ZHANG Y, LI K, LI K, et al. Image super-resolution using very deep residual channel attention networks[C]//Proceedings of the European Conference on Computer Vision (ECCV), 2018: 286-301.
[17]	TONG T, LI G, LIU X, et al. Image super-resolution using dense skip connections[C]//Proceedings of the IEEE International Conference on Computer Vision, 2017: 4799-4807.
[18]	ZHANG K, Liang J, Van Gool L, et al. Designing a practical degradation model for deep blind image super-resolution[C]//Proceedings of the IEEE/CVF International Conference on Computer Vision, 2021: 4791-4800.
[19]	WANG X, XIE L, DONG C, et al. Real-esrgan: Training real-world blind super-resolution with pure synthetic data[C]//Proceedings of the IEEE/CVF International Conference on Computer Vision, 2021: 1905-1914.
[20]	ZHANG W, SHI G, LIU Y, et al. A closer look at blind super-resolution: Degradation models, baselines, and performance upper bounds[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2022: 527-536.
[21]	LIANG J, CAO J, SUN G, et al. Swinir: Image restoration using swin transformer[C]//Proceedings of the IEEE/CVF International Conference on Computer Vision, 2021: 1833-1844.
[22]	Huynh-Thu Q, Ghanbari M. Scope of validity of PSNR in image/video quality assessment[J]. Electronics Letters, 2008, 44(13): 800-801. doi: 10.1049/el:20080522
[23]	Hanhart P, Korshunov P, Ebrahimi T. Benchmarking of quality metrics on ultra-high definition video sequences[C]//18th International Conference on Digital Signal Processing (DSP)of IEEE, 2013: 1-8.
[24]	Kundu D, Evans B L. Full-reference visual quality assessment for synthetic images: A subjective study[C]// IEEE International Conference on Image Processing (ICIP), 2015: 2374-2378.
[25]	Mittal A, Soundararajan R, Bovik A C. Making a "completely blind" image quality analyzer[J]. IEEE Signal Processing Letters, 2012, 20(3): 209-212.
[26]	Mittal A, Moorthy A K, Bovik A C. No-reference image quality assessment in the spatial domain[J]. IEEE Transactions on Image Processing, 2012, 21(12): 4695-4708.
[27]	Blau Y, Mechrez R, Timofte R, et al. The 2018 PIRM challenge on perceptual image super-resolution[C]//Proceedings of the European Conference on Computer Vision (ECCV), 2018: 334-355.