| Citation: |
LIAO Guangfeng, GUAN Zhiwei, CHEN Qiang. An Improved Dual Discriminator Generative Adversarial Network Algorithm for Infrared and Visible Image Fusion[J]. Infrared Technology , 2025, 47(3): 367-375.
|
An infrared and visible image fusion algorithm, based on a dual-discriminator generative adversarial network, is proposed to address issues, such as the insufficient extraction of global and multiscale features and the imprecise extraction of key information, in existing infrared and visible image fusion algorithms. First, a generator combines convolution and self-attention mechanisms to capture multiscale local and global features. Second, the attention mechanism is combined with skip connections to fully utilize multiscale features and reduce information loss during the downsampling process. Finally, two discriminators guide the generator to focus on the salient targets of the infrared images and background texture information of visible-light images, allowing the fused image to retain more critical information. Experimental results on the public multi-scenario multi-modality (M3FD) and multi-spectral road scenarios (MSRS) datasets show that compared with the baseline algorithms, the results of the six evaluation metrics improved significantly. Specifically, the average gradient (AG) increased by 27.83% and 21.06% on the two datasets, respectively, compared with the second-best results. The fusion results of the proposed algorithm are rich in detail and exhibit superior visual effects.
| [1] |
ZHANG H, XU H, TIAN X, et al. Image fusion meets deep learning: a survey and perspective[J]. Information Fusion, 2021, 76: 323-336. DOI: 10.1016/j.inffus.2021.06.008
|
| [2] |
谭明川, 聂仁灿, 张谷铖, 等. 基于深度学习的红外与可见光图像融合综述[J]. 云南大学学报(自然科学版), 2023, 45(2): 326-343.
TAN M, NIE R, ZHANG G, et al. A review of infrared and visible image fusion based on deep learning[J]. Journal of Yunnan University (Natural Science Edition), 2023, 45(2): 326-343.
|
| [3] |
JIAN L, YANG X, LIU Z, et al. SEDRFuse: A symmetric encoder–decoder with residual block network for infrared and visible image fusion[J]. IEEE Transactions on Instrumentation and Measurement, 2020, 70: 1-15.
|
| [4] |
TANG L, YUAN J, ZHANG H, et al. PIAFusion: a progressive infrared and visible image fusion network based on illumination aware[J]. Information Fusion, 2022, 83: 79-92.
|
| [5] |
MA J, TANG L, XU M, et al. STDFusionNet: an infrared and visible image fusion network based on salient target detection[J]. IEEE Transactions on Instrumentation and Measurement, 2021, 70: 1-13.
|
| [6] |
Goodfellow I, Pouget Abadie J, Mirza M, et al. Generative adversarial nets[J/OL]. Advances in Neural Information Processing Systems, 2014: 2672-2680, https://arxiv.org/abs/1406.2661.
|
| [7] |
LIU J, FAN X, HUANG Z, et al. Target-aware dual adversarial learning and a multi-scenario multi-modality benchmark to fuse infrared and visible for object detection[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2022: 5802-5811.
|
| [8] |
RAO D, XU T, WU X J. TGFuse: An infrared and visible image fusion approach based on transformer and generative adversarial network[J/OL]. IEEE Transactions on Image Processing, 2023, Doi: 10.1109/TIP.2023.3273451.
|
| [9] |
HUANG Z, WANG X, HUANG L, et al. Ccnet: Criss-cross attention for semantic segmentation[C]//Proceedings of the IEEE/CVF International Conference on Computer Vision, 2019: 603-612.
|
| [10] |
ZHAO H, KONG X, HE J, et al. Efficient image super-resolution using pixel attention[C]//Computer Vision–ECCV, 2020: 56-72.
|
| [11] |
Ronneberger O, Fischer P, Brox T. U-net: Convolutional networks for biomedical image segmentation[C]//Medical Image Computing and Computer-Assisted Intervention–MICCAI, 2015: 234-241.
|
| [12] |
Sandler M, Howard A, ZHU M, et al. Mobilenetv2: Inverted residuals and linear bottlenecks[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2018: 4510-4520.
|
| [13] |
ZHANG Y, TIAN Y, KONG Y, et al. Residual dense network for image super-resolution[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2018: 2472-2481.
|
| [14] |
SHI W, Caballero J, Huszár F, et al. Real-time single image and video super-resolution using an efficient sub-pixel convolutional neural network[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2016: 1874-1883.
|
| [15] |
QIN X, ZHANG Z, HUANG C, et al. U2-Net: Going deeper with nested U-structure for salient object detection[J]. Pattern Recognition, 2020, 106: 107404. DOI: 10.1016/j.patcog.2020.107404
|
| [16] |
MA J, XU H, JIANG J, et al. DDcGAN: A dual-discriminator conditional generative adversarial network for multi-resolution image fusion[J]. IEEE Transactions on Image Processing, 2020, 29: 4980-4995. DOI: 10.1109/TIP.2020.2977573
|
| [17] |
LI H, WU X J. DenseFuse: A fusion approach to infrared and visible images[J]. IEEE Transactions on Image Processing, 2018, 28(5): 2614-2623.
|
| [18] |
LI H, XU T, WU X J, et al. Lrrnet: A novel representation learning guided fusion network for infrared and visible images[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2023, 45(9): 11040-11052. DOI: 10.1109/TPAMI.2023.3268209
|
| [19] |
LI H, WU X J, Durrani T. NestFuse: An infrared and visible image fusion architecture based on nest connection and spatial/channel attention models[J]. IEEE Transactions on Instrumentation and Measurement, 2020, 69(12): 9645-9656. DOI: 10.1109/TIM.2020.3005230
|
| [20] |
LI H, WU X J, Kittler J. RFN-Nest: An end-to-end residual fusion network for infrared and visible images[J]. Information Fusion, 2021, 73: 72-86. DOI: 10.1016/j.inffus.2021.02.023
|
| [21] |
TANG L, YUAN J, MA J. Image fusion in the loop of high-level vision tasks: A semantic-aware real-time infrared and visible image fusion network[J]. Information Fusion, 2022, 82: 28-42. DOI: 10.1016/j.inffus.2021.12.004
|
| [1] | YUAN Xilin, ZHANG Baohui, ZHANG Qian, HE Ming, ZHOU Jinjie, LIAN Cheng, YUE Jiang. Infrared Images with Super-resolution Based on Deep Convolutional Neural Network[J]. Infrared Technology , 2023, 45(5): 498-505. |
| [2] | CAO Yutong, HUAN Kewei, XUE Chao, HAN Fengdi, LI Xiangyang, CHEN Xiao. Infrared and Visible Image Fusion Based on CNN with NSCT[J]. Infrared Technology , 2023, 45(4): 378-385. |
| [3] | WANG Kun, SHI Yong, LIU Chichi, XIE Yi, CAI Ping, KONG Songtao. A Review of Infrared Spectrum Modeling Based on Convolutional Neural Networks[J]. Infrared Technology , 2021, 43(8): 757-765. |
| [4] | LIN Li, LIU Xin, ZHU Junzhen, FENG Fuzhou. Classification of Ultrasonic Infrared Thermal Images Using a Convolutional Neural Network[J]. Infrared Technology , 2021, 43(5): 496-501. |
| [5] | CHEN Gao, WANG Weihua, LIN Dandan. Infrared Vehicle Target Detection Based on Convolutional Neural Network without Pre-training[J]. Infrared Technology , 2021, 43(4): 342-348. |
| [6] | DONG Anyong, DU Qingzhi, SU Bin, ZHAO Wenbo, YU Wen. Infrared and Visible Image Fusion Based on Convolutional Neural Network[J]. Infrared Technology , 2020, 42(7): 660-669. |
| [7] | JING Fusheng, LI Peng, JIANG Haijun, CHEN Li, WEI Yibing. Crack Detection of Locomotive Hook Tongue Based on Ultrasonic Thermography[J]. Infrared Technology , 2020, 42(2): 158-162. |
| [8] | WANG Pengxiang, GUO Jingbin, TAN Wenbin, LI Xingfei. A Multidomain CNN that Integrates Multiple Models in a Tree Structure for Visual Tracking[J]. Infrared Technology , 2018, 40(1): 47-54. |
| [9] | MIN Zhaoyang, ZHAO Wenjie. A Target Tracking Algorithm Combining Convolution Neural Network with Spatio Temporal Context[J]. Infrared Technology , 2017, 39(8): 740-745. |
| [10] | WANG Chen, TANG Xinyi, GAO Sili. Infrared Scene Understanding Algorithm Based on Deep Convolutional Neural Network[J]. Infrared Technology , 2017, 39(8): 728-733. |
| 1. |
谭何盛，陈明欣，赵恒，王玲燕，杨文波，邓华兵，靳英坤，冯云祥，刀丽纯，张昆林. EBAPS器件中陶瓷基座与CMOS硅基芯片的粘接. 红外技术. 2025(05): 571-577 .
 本站查看
| |
| 2. |
闫磊，石峰，程宏昌，焦岗成，杨晔，肖超，樊海波，郑舟，董海晨，何惠洋. 低能电子轰击引起氧化铝钝化膜BCMOS传感器暗电流变化研究. 红外技术. 2024(03): 342-346 .
本站查看
| |
| 3. |
韩剑，焦岗成，闫磊，程宏昌，杨晔，雷石伟，樊海波，李桐桐. 数字微光器件研究进展. 应用光学. 2023(04): 874-886 .
| |
| 4. |
李书涵，陈文娥，王重霄，陈卫军，宋德，李野. EBCMOS钝化层表面残气吸附特性. 半导体光电. 2023(06): 901-906 .
| |
| 5. |
王熠，苗凯. 红外遥感探测下运动姿态多目标图像重构方法. 激光杂志. 2020(06): 89-93 .
| |
| 6. |
张红. 基于现实增强技术的数字绘本图像传感器设计. 自动化与仪器仪表. 2020(09): 84-87 .
|
Supported by: Beijing Renhe Information Technology Co., Ltd. 
DownLoad: