| Citation: |
LIU Xiaopeng, ZHANG Tao. Global-Local Attention-Guided Reconstruction Network for Infrared Image[J]. Infrared Technology , 2024, 46(7): 791-801.
|
To solve the problems of image blur smoothing, texture distortion, and excessively large parameters in real-world infrared-image recovery algorithms, a global-local attention-guided super-resolution reconstruction algorithm for infrared images is proposed. First, a cross-scale global-local feature fusion module utilizes multi-scale convolution and a transformer to fuse information at different scales in parallel and to guide the effective fusion of global and local information by learnable factors. Second, a novel domain randomization degradation model accommodates the degradation domain of real-world infrared images. Finally, a new hybrid loss based on weight learning and regularization penalty enhances the recovery capability of the network while speeding up convergence. Test results on classical degraded images and real-world infrared images show that, compared with existing methods, the images recovered by the proposed algorithm have more realistic textures and fewer boundary artifacts. Moreover, the total number of parameters can be reduced by up to 20%.
| [1] |
胡德敏, 闵天悦. 改进型轻量级GAN的红外图像超分辨率方法[J]. 小型微型计算机系统, 2022, 43(8): 1711-1717. https://www.cnki.com.cn/Article/CJFDTOTAL-XXWX202208021.htm
HU Demin, MIN Tianyue. Infrared image super-resolution via improved lightweight GAN[J]. Journal of Chinese Computer Systems, 2022, 43(8): 1711-1717. https://www.cnki.com.cn/Article/CJFDTOTAL-XXWX202208021.htm
|
| [2] |
MEI Y, FAN Y, ZHOU Y. Image super-resolution with non-local sparse attention[C]//Proc. CVPR, 2021: 3517-3526. DOI: 10.1109/CVPR46437.2021.00352.
|
| [3] |
马敏慧, 王红茹, 王佳. 基于改进的MSRCR-CLAHE融合的水下图像增强算法[J]. 红外技术, 2023, 45(1): 23-32. http://hwjs.nvir.cn/cn/article/id/69e5b90e-9c0c-43c6-b4e2-dedede3eb414
MA Minhui, WANG Hongru, WANG Jia. An underwater image enhancement algorithm based on improved MSRCR-CLAHE fusion[J]. Infrared Technology, 2023, 45(1): 23-32. http://hwjs.nvir.cn/cn/article/id/69e5b90e-9c0c-43c6-b4e2-dedede3eb414
|
| [4] |
ZHANG D, SHAO J, LI X, et al. Remote sensing image super-resolution via mixed high-order attention network[J]. IEEE Trans. Geosci. Remote Sens. , 2021, 59(6): 5183-5196. DOI: 10.1109/TGRS.2020.3009918
|
| [5] |
刘云峰, 赵洪山, 杨晋彪, 等. 基于GNR先验的电力设备热成像超分辨率方法[J]. 红外技术, 2023, 45(1): 40-48. http://hwjs.nvir.cn/cn/article/id/3f88d6d0-ab5c-4cd7-999e-b50ffce93699
LIU Yunfeng, ZHAO Hongshan, YANG Jinbiao, et al. Super resolution method for power equipment infrared imaging based on gradient norm-ratio prior[J]. Infrared Technology, 2023, 45(1): 40-48. http://hwjs.nvir.cn/cn/article/id/3f88d6d0-ab5c-4cd7-999e-b50ffce93699
|
| [6] |
DONG C, Loy C, HE K, et al. Learning a deep convolutional network for image super-resolution[C]//Proc. Computer Vision, ECCV, 2014: 184-199. DOI: 10.1007/978-3-319-10593-2_13.
|
| [7] |
聂丰英, 侯利霞, 万里勇. 自适应双边滤波与方向梯度的红外图像增强[J]. 红外技术, 2022, 44(12): 1309-1315. http://hwjs.nvir.cn/cn/article/id/8fbb7847-4550-463c-a220-9e97cd402968
NIE Fengying, HOU Lixia, WAN Liyong. Infrared image enhancement based on adaptive bilateral filtering and directional gradient[J]. Infrared Technology, 2022, 44(12): 1309-1315. http://hwjs.nvir.cn/cn/article/id/8fbb7847-4550-463c-a220-9e97cd402968
|
| [8] |
Timofte R, Rothe R, Van Gool L. Seven ways to improve example-based single image super resolution[C]//Proc. CVPR, 2016: 1865-1873. DOI: 10.1109/CVPR.2016.206
|
| [9] |
Lim B, Son S, Kim H, et al. Enhanced deep residual networks for single image super-resolution[C]//Proc. CVPRW, 2017: 136-144. Doi: 10.1109/CVPRW.2017.151
|
| [10] |
Bevilacqua M, Roumy A, Guillemot C, et al. Low-complexity single-image super-resolution based on nonnegative neighbor embedding[C]// Proc. BMVC, 2012: 135-141. DOI: 10.5244/C.26.135.
|
| [11] |
TANG Y, GONG W, CHEN X, et al. Deep inception-residual Laplacian pyramid networks for accurate single-image super-resolution[J]. IEEE Transactions on Neural Networks and Learning Systems, 2019, 31(5): 1514-1528.
|
| [12] |
CAI J, ZENG H, YONG H, et al. Toward real-world single image super-resolution: a new benchmark and a new model[C]//Proc. ICCV, 2019: 3086-3095. DOI: 10.1109/ICCV.2019.00318.
|
| [13] |
SHI W, Caballero J, Huszár F, et al. Real-time single image and video super-resolution using an efficient sub-pixel convolutional neural network[C]//Proc. CVPR, 2016: 1874-1883. DOI: 10.1109/CVPR.2016.207.
|
| [14] |
ZHANG Y, TIAN Y, KONG Y, et al. Residual dense network for image super-resolution[C]//Proc. CVPR, 2018: 2472-2481. DOI: 10.1109/CVPR.2018.00262.
|
| [15] |
Esser P, Rombach R, Ommer B. Taming transformers for high-resolution image synthesis[C]//Proc. CVPR, 2021: 12873-12883. DOI: 10.1109/CVPR46437.2021.01268.
|
| [16] |
JIANG Y, CHANG S, WANG Z. Transgan: two transformers can make one strong GAN[J/OL]. Computer Vision and Pattern Recognition, 2021, 34: https://arxiv.org/abs/2102.07074.
|
| [17] |
ZHANG K, LIANG J, Van Gool L, et al. Designing a practical degradation model for deep blind image super-resolution[C]//Proc. ICCV, 2021: 4771-4780. DOI: 10.1109/ICCV48922.2021.00475.
|
| [18] |
WANG X, XIE L, DONG C, et al. Real-ESRGAN: Training real-world blind super-resolution with pure synthetic data[C]//Proc. ICCVW, 2021: 1905-1914. DOI: 10.1109/ICCVW54120.2021.00217
|
| [19] |
Ledig C, Theis L, Huszar F, et al. Photo-realistic single image super-resolution using a generative adversarial network[C]//Proc. CVPR, 2017: 4681-4690. DOI: 10.1109/CVPR.2017.19.
|
| [20] |
ZHOU Y, WU G, FU Y, et al. Cross-MPI: cross-scale stereo for image super-resolution using multiplane images[C]//Proc. CVPR, 2021: 14842-14851. DOI: 10.1109/CVPR46437.2021.01460.
|
| [21] |
烟台艾睿光电科技有限公司. 艾睿光电红外开源数据库[EB/OL]. [2023-02-26]. http://openai.raytrontek.com/apply/Super_resolution.html/.
IRay Technology Co., Ltd. IRay Optoelectronic Infrared Open Source Database[EB/OL]. [2023-02-26]. http://openai.raytrontek.com/apply/Super_resolution.html/.
|
| [22] |
ZHANG Y, LI K, LI K, et al. Image super-resolution using very deep residual channel attention networks[C]//Proc. ECCV, 2018: 286-301. DOI: 10.1007/978-3-030-01234-2_18.
|
| [23] |
NIU B, WEN W, REN W, et al. Single image super-resolution via a holistic attention network[C]//Proc. ECCV, 2020, 12357: 191-207.
|
| [24] |
WANG X, YU K, WU S, et al. ESRGAN: Enhanced super-resolution generative adversarial networks[C]// Proc. ECCV, 2019: 63-79. DOI: 10.1007/978-3-030-11021-5_5.
|
| [25] |
LIU Z, LIN Y, CAO Y, et al. Swin transformer: Hierarchical vision transformer using shifted windows[C]//Proc. ICCV, 2021: 9992-10002. DOI: 10.1109/ICCV48922.2021.00986.
|
| [26] |
WANG Y, WANG L, WANG H, et al. Resolution-aware network for image super-resolution[J]. IEEE Transactions on Circuits and Systems for Video Technology, 2018, 29(5): 1259-1269.
|
| [27] |
Andreas Lugmayr, Martin Danelljan, Radu Timofte. Unsupervised learning for real-world super-resolution[J]. ICCV Workshop, 2019, 2(3): 3408-3416.
|
| [28] |
DAI T, CAI J, ZHANG Y, et al. Second-order attention network for single image super-resolution[C]//Proc. CVPR, 2019: 11057-11066. DOI: 10.1109/CVPR.2019.01132.
|
| [29] |
Agustsson E, Timofte R. Ntire 2017 challenge on single image super-resolution: Dataset and study[C]// Proc. CVPRW, 2017: 126-135. DOI: 10.1109/CVPRW.2017.150.
|
| [30] |
LIANG J, CAO J, SUN G, et al. SwinIR: Image restoration using swin transformer[C]//Proceedings of the IEEE/CVF International Conference on Computer Vision, 2021: 1833-1844. DOI: 10.1109/ICCVW54120.2021.00210.
|
| [31] |
Blau Y, Mechrez R, Timofte R, et al. The 2018 PIRM challenge on perceptual image super-resolution[C]//Proceedings of the European Conference on Computer Vision (ECCV), 2018, DOI: 10.1007/978-3-030-11021-5_21
|
| [32] |
MA C, YANG C Y, YANG X, et al. Learning a no-reference quality metric for single-image super-resolution[J]. Computer Vision and Image Understanding, 2017, 158: 1-16. DOI: 10.1016/j.cviu.2016.12.009
|
| [1] | ZHAO Yating, HAN Long, HE Huihuang, CHEN Chu. DSEL-CNN: Image Fusion Algorithm Combining Attention Mechanism and Balanced Loss[J]. Infrared Technology , 2025, 47(3): 358-366. |
| [2] | LI Ruihong, FU Zhitao, ZHANG Shaochen, ZHANG Jian, WANG Leiguang. Nighttime Object Detection in Infrared and Visible Images Based on Multi-Attention Mechanism[J]. Infrared Technology , 2024, 46(12): 1371-1379. |
| [3] | QI Yanjie, HOU Qinhe. Infrared and Visible Image Fusion Combining Multi-scale and Convolutional Attention[J]. Infrared Technology , 2024, 46(9): 1060-1069. |
| [4] | DI Jing, LIANG Chan, REN Li, GUO Wenqing, LIAN Jing. Infrared and Visible Image Fusion Based on Multi-Scale Contrast Enhancement and Cross-Dimensional Interactive Attention Mechanism[J]. Infrared Technology , 2024, 46(7): 754-764. |
| [5] | CHONG Fating, DONG Zhangyu, YANG Xuezhi, ZENG Qingwang. SAR and Multispectral Image Fusion Based on Dual-channel Multi-scale Feature Extraction and Attention[J]. Infrared Technology , 2024, 46(1): 61-73. |
| [6] | XU Haiyang, ZHAO Wei, LIU Jianye. Infrared and Visible Image Registration Algorithm Based on Edge Structure Features[J]. Infrared Technology , 2023, 45(8): 858-862. |
| [7] | HE Zifen, CAO Huizhu, ZHANG Yinhui, HUANG Junxuan, SHI Benjie, ZHU Shouye. Infrared Image Segmentation of Methane Leaks Incorporating Attentional Branching Features[J]. Infrared Technology , 2023, 45(4): 417-426. |
| [8] | WU Jie, MA Xiaohu. Anti-Occlusion Infrared Target Tracking AlgorithmBased on Fusion of Discriminant and Fine-Grained Features[J]. Infrared Technology , 2022, 44(11): 1139-1145. |
| [9] | WANG Fang, LI Chuanqiang, WU Bo, YU Kun, JIN Chan, CHEN Yake, LU Yinghui. Infrared Small Target Detection Method Based on Multi-Scale Feature Fusion[J]. Infrared Technology , 2021, 43(7): 688-695. |
| [10] | HUO Yuanlian, QIN Mei, SONG Yali. Moving Target Detection Method Based on Edge Character and Multiple Frame Difference[J]. Infrared Technology , 2017, 39(2): 157-162. |
Supported by: Beijing Renhe Information Technology Co., Ltd. 
DownLoad: