Electronic Zooming of Infrared Image Based on Lightweight Multi-scale Aggregation Network
-
摘要:
为了解决光电领域中低分辨率红外图像影响观瞄的问题，构建一种轻量级多尺度聚合网络算法来增强电子变倍时中心区域图像分辨率，该算法首先使用不同大小的尺度核从图像中提取特征信息，并利用浅层残差结构将局部多尺度残差特征有效聚合在一起，以获得更强大的特征表示能力；然后采用基于对比度感知的通道注意层来聚合更多尺度特征信息，最终重构出具有丰富细节而清晰的高分辨率红外图像。仿真实验结果表明，本文所提出的电子变倍方法在不引入额外参数的情况下能够提取出精细的多尺度特征信息，获得清晰的重建效果。
Abstract:To solve the problem of low-resolution infrared images affecting viewing and aiming in the photoelectric field, a lightweight multi-scale aggregation network is proposed to enhance the resolution of the central region when the IR image is zoomed. First, the algorithm uses scale kernels of different sizes to extract feature information and employs a shallow residual structure to effectively aggregate local multi-scale residual features, thereby obtaining stronger feature representation capability. Then, a channel attention layer based on contrast perception is used to aggregate more multi-scale feature information. Finally, a high-resolution infrared image with rich detail and clarity is reconstructed. Simulation results show that the zooming method can extract fine multi-scale feature information without introducing additional parameters and can produce clear reconstruction results.
-
Keywords:
- photoelectric system /
- infrared image /
- electronic zooming /
- multi-scale aggregation /
- contrast-aware /
- lightweight
-
-
![]()
图 4 不同多尺度聚合模块数量的性能对比
Figure 4. Performance comparison for different number of multi-scale aggregation modules
![]()
图 5 2×变倍模式下不同模型的重建结果
Figure 5. Visual results for a scale factor of 2×on the different models
![]()
图 6 3×变倍模式下不同模型的重建结果
Figure 6. Visual results for a scale factor of 3× on the different models
![]()
图 7 4×变倍模式下不同模型的重建结果
Figure 7. Visual results for a scale factor of 4×on the different models
![]()
图 8 不同算法的重建差值图对比
Figure 8. Comparison of reconstruction difference maps of different models
表 1 典型图像及其测试数据集的超分辨重建结果对比（PSNR(dB)/SSIM），其中黑色加粗表示最优结果
Table 1 Comparison of super-resolution reconstruction results for typical images and their test-data sets, where black represents the best result
Dataset Scale Bicubic SRCNN VDSR EDSR GAN-SR Proposed img_01 2× 33.12/0.9219 35.73/0.9544 37.52/0.9522 38.17/0.96000 38.08/95.35 38.17/0.9631 3× 30.38/0.8682 32.57/0.9087 33.67/0.9215 34.56/0.9248 34.77/0.9301 34.79/0.9287 4× 28.41/0.8102 30.28/0.8709 31.35/0.8838 32.15/0.8917 31.82/0.8991 32.11/0.9011 img_02 2× 29.56/0.8432 32.35/0.9045 33.03/0.9124 33.87/0.9153 33.85/0.9199 33.91/0.9201 3× 27.20/0.7384 30.110.8726 29.78/0.8425 30.25/0.8954 30.45/0.8452 30.53/0.8467 4× 25.92/0.6679 27.48/0.7427 28.11/0.7701 28.75/0.7846 28.71/0.7911 28.91/0.7912 img_03 2× 26.87/0.8403 29.12/0.8965 30.76/0.9144 32.49/0.9384 32.84/0.9125 33.02/0.936233 3× 24.69/0.7439 25.99/0.8075 27.01/0.8257 28.75/0.8647 28.78/0.8655 28.95/0.8702 4× 23.28/0.6544 24.81/0.7204 25.22/0.7528 26.62/0.8034 26.68/0.8017 26.71/0.8034 DIV2K 2× 31.24/0.9387 33.24/0.9451 33.38/0.9614 34.95/0.9648 35.25/0.9684 35.35/0.9641 3× 28.12/0.8907 29.64/0.9127 30.14/0.9215 31.26/0.9341 31.29/0.9354 31.42/0.9354 4× 26.58/0.8486 27.74/0.8705 28.14/0.8803 29.35/0.9014 29.38/0.9011 29.24/0.9024 Self-built 2× 30.15/0.9217 31.53/0.9197 31.82/0.9201 32.07/0.92971 32.21/0.9311 32.32/0.9308 3× 27.15/0.8295 28.24/0.8336 29.11/0.9012 29.25/0.8924 29.29/0.8174 29.67/0.8947 4× 26.64/0.8074 27.14/0.8391 27.32/0.8765 27.71/0.8607 27.86/0.8653 27.84/0.8724 
下载: 导出CSV
-
[1] 邓伟, 陈建飞, 张胜. 深层跳线残差网络热红外图像超分辨重建[J]. 电光与控制, 2023, 30(3): 27-32. DENG Wei, CHEN Jianfei, ZHANG Sheng. Super-resolution reconstruction of thermal infrared images based on deep skip connection residual network[J]. Electronics Optics and Control, 2023, 30(3): 27-32.
[2] 唐中和, 王升哲, 霍建亮, 等. 基于局部结构变换域稀疏外观模型的目标跟踪[J]. 电视技术, 2017, 41(Z2): 140-146. TANG Zhonghe, WANG Shengzhe, HUO Jianliang, et al. The target tracking based on sparse appearance model of local structure transformed domain[J]. The TV Engineering, 2017, 41(Z2): 140-146.
[3] 左岑, 杨秀杰, 张捷, 等. 基于轻量级金字塔密集残差网络的红外图像超分辨增强[J]. 红外技术, 2021, 43(3): 251-257. http://hwjs.nvir.cn/article/id/a1540103-27df-466a-a058-1fd126ff5aec ZUO Cen, YANG Xiujie, ZHANG Jie, et al. The super-resolution enhancement of the infrared images based on the lightweight and pyramid of dense residual network[J]. Infrared Technology, 2021, 43(3): 251-257. http://hwjs.nvir.cn/article/id/a1540103-27df-466a-a058-1fd126ff5aec
[4] ZHANG X, LI C, MENG Q, et al. Infrared image super resolution by combining compressive sensing and deep learning[J]. Sensors, 2018, 18(8): 2587. DOI: 10.3390/s18082587
[5] ZOU Y, ZHANG L, LIU C, et al. Super-resolution reconstruction of infrared images based on a convolutional neural network with skip connections[J]. Optics and Lasers in Engineering, 2021, 146: 106717. DOI: 10.1016/j.optlaseng.2021.106717
[6] HUANG Y, JIANG Z, LAN R, et al. Infrared image super-resolution via transfer learning and PSRGAN[J]. IEEE Signal Processing Letters, 2021, 28: 982-986. DOI: 10.1109/LSP.2021.3077801
[7] Mallat S, YU G. Super-resolution with sparse mixing estimators[J]. IEEE Transactions on Image Processing, 2010, 19(11): 2889-2900. DOI: 10.1109/TIP.2010.2049927
[8] DONG C, Loy C C, HE K, et al. Image super-resolution using deep convolutional networks[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2016, 38(2): 295-307. DOI: 10.1109/TPAMI.2015.2439281
[9] 杨明, 王璇, 高宏伟. 基于多级深度网络的高清晰度红外电子变倍算法[J]. 弹箭与制导学报, 2020, 40(4): 14-19. YANG Ming, WANG Xuan, GAO Hongwei. The high-definition infrared electronic zoom algorithm based on multi-level deep network[J]. Journal of Projectiles, Rockets, Missiles and Guidance, 2020, 40(4): 14-19.
[10] CHEN X, WANG X, ZHOU J, et al. Activating more pixels in image super-resolution transformer[C]// IEEE Conf. Comput. Vis. Pattern Recognit. (CVPR), 2023: 236-248.
[11] LIANG J, ZENG H, ZHANG L. Details or artifacts: a locally discriminative learning approach to realistic image super-resolution[C]//2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2022: 5647-5656.
[12] HE Z, TANG S, YANG J, et al. Cascaded deep networks with multiple receptive fields for infrared image super-resolution[J]. IEEE Transactions on Circuits and Systems for Video Technology, 2018, 29(8): 2310-2322.
[13] 左芝勇. 采用分裂Bregman的空间邻近目标红外超分辨算法[J]. 电讯技术, 2020, 60(7): 778-784. ZUO Zhiyong. An infrared super-resolution algorithm using split bregman for closely spaced objects[J]. Telecommunication Engineering, 2020, 60(7): 778-784.
[14] QIN J, HUANGY, WEN W. Multi-scale feature fusion residual network for single image super-resolution[J]. Neuro Computing, 2020, 379: 334-342.
[15] HAI H, LI P, ZOU N, et al. End-to-end super-resolution for remote-sensing images using an improved multi-scale residual network[J]. Remote Sensing, 2021, 13(4): 666-672.
[16] Szegedy C, LIU W, JIA Y, et al. Going deeper with convolutions[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2015: 1-9.
[17] WANG Yingqian, YING Xinyi, WANG Longguang, et al. Light field super-resolution via adaptive feature remixing[J]. European Conference on Computer Vision, 2022: 230-247. http://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=9394760
[18] CHEN C, GONG D, WANG H, et al. Learning spatial attention for face super-resolution[J]. IEEE Transactions on Image Processing, 2021, 30: 1219-1231.
[19] HU Jie, SHEN Li, SUN Gang. Deep convolutional neural networks with merge-and-run mappings[C]//International Journal of Computer Vision, 2018: 1379-1398.
[20] LIU Zhiwei, ZHOU Lei, GAO Yuting, et al. Large-scale object detection in the wild from imbalanced multi-labels[C]//IEEE Transactions on Image Processing, 2022: 1234-1247.
[21] HUI Z, GAO X, YANG Y, et al. Lightweight image super-resolution with information multi-distillation network[C]//Proceedings of the 27th ACM International Conference on Multimedia, 2019: 2024-2032.
[22] LI Y, CAO J, LI Z, et al. Lightweight single image super-resolution with dense connection distillation network[J]. ACM Transactions on Multimedia Computing, Communications, and Applications (TOMM), 2021, 17(1s): 1-17.
[23] Agustsson E, Timofte R. Ntire 2017 challenge on single image super-resolution: Dataset and study[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, 2017: 126-135.
[24] YANG W, ZHANG X, TIAN Y, et al. Deep learning for single image super-resolution: a brief review[J]. IEEE Transactions on Multimedia, 2019, 21(12): 3106-3121. http://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=8723565
[25] ZHANG Kai, LIANG Jingyun, Van Gool Luc, et al. Training real-world blind super-resolution with pure synthetic data[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2022: 1401-1411.
[26] Kim J, Lee J K, Lee K M. Accurate image super-resolution using very deep convolutional networks[C]//IEEE Conf. Comput. Vis. Pattern Recognit. (CVPR), 2016: 1646-1654.
[27] Lim B, Son S, Kim H, et al. Enhanced deep residual networks for single image super-resolution[C]//IEEE Conf. Comput. Vis. Pattern Recognit. (CVPR), 2017: 1025-1032.
计量
- 文章访问数: 44
- HTML全文浏览量: 6
- PDF下载量: 23
