| Citation: |
LIAO Guangfeng, GUAN Zhiwei, CHEN Qiang. An Improved Dual Discriminator Generative Adversarial Network Algorithm for Infrared and Visible Image Fusion[J]. Infrared Technology , 2025, 47(3): 367-375.
|
An infrared and visible image fusion algorithm, based on a dual-discriminator generative adversarial network, is proposed to address issues, such as the insufficient extraction of global and multiscale features and the imprecise extraction of key information, in existing infrared and visible image fusion algorithms. First, a generator combines convolution and self-attention mechanisms to capture multiscale local and global features. Second, the attention mechanism is combined with skip connections to fully utilize multiscale features and reduce information loss during the downsampling process. Finally, two discriminators guide the generator to focus on the salient targets of the infrared images and background texture information of visible-light images, allowing the fused image to retain more critical information. Experimental results on the public multi-scenario multi-modality (M3FD) and multi-spectral road scenarios (MSRS) datasets show that compared with the baseline algorithms, the results of the six evaluation metrics improved significantly. Specifically, the average gradient (AG) increased by 27.83% and 21.06% on the two datasets, respectively, compared with the second-best results. The fusion results of the proposed algorithm are rich in detail and exhibit superior visual effects.
| [1] |
ZHANG H, XU H, TIAN X, et al. Image fusion meets deep learning: a survey and perspective[J]. Information Fusion, 2021, 76: 323-336. DOI: 10.1016/j.inffus.2021.06.008
|
| [2] |
谭明川, 聂仁灿, 张谷铖, 等. 基于深度学习的红外与可见光图像融合综述[J]. 云南大学学报(自然科学版), 2023, 45(2): 326-343.
TAN M, NIE R, ZHANG G, et al. A review of infrared and visible image fusion based on deep learning[J]. Journal of Yunnan University (Natural Science Edition), 2023, 45(2): 326-343.
|
| [3] |
JIAN L, YANG X, LIU Z, et al. SEDRFuse: A symmetric encoder–decoder with residual block network for infrared and visible image fusion[J]. IEEE Transactions on Instrumentation and Measurement, 2020, 70: 1-15.
|
| [4] |
TANG L, YUAN J, ZHANG H, et al. PIAFusion: a progressive infrared and visible image fusion network based on illumination aware[J]. Information Fusion, 2022, 83: 79-92.
|
| [5] |
MA J, TANG L, XU M, et al. STDFusionNet: an infrared and visible image fusion network based on salient target detection[J]. IEEE Transactions on Instrumentation and Measurement, 2021, 70: 1-13.
|
| [6] |
Goodfellow I, Pouget Abadie J, Mirza M, et al. Generative adversarial nets[J/OL]. Advances in Neural Information Processing Systems, 2014: 2672-2680, https://arxiv.org/abs/1406.2661.
|
| [7] |
LIU J, FAN X, HUANG Z, et al. Target-aware dual adversarial learning and a multi-scenario multi-modality benchmark to fuse infrared and visible for object detection[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2022: 5802-5811.
|
| [8] |
RAO D, XU T, WU X J. TGFuse: An infrared and visible image fusion approach based on transformer and generative adversarial network[J/OL]. IEEE Transactions on Image Processing, 2023, Doi: 10.1109/TIP.2023.3273451.
|
| [9] |
HUANG Z, WANG X, HUANG L, et al. Ccnet: Criss-cross attention for semantic segmentation[C]//Proceedings of the IEEE/CVF International Conference on Computer Vision, 2019: 603-612.
|
| [10] |
ZHAO H, KONG X, HE J, et al. Efficient image super-resolution using pixel attention[C]//Computer Vision–ECCV, 2020: 56-72.
|
| [11] |
Ronneberger O, Fischer P, Brox T. U-net: Convolutional networks for biomedical image segmentation[C]//Medical Image Computing and Computer-Assisted Intervention–MICCAI, 2015: 234-241.
|
| [12] |
Sandler M, Howard A, ZHU M, et al. Mobilenetv2: Inverted residuals and linear bottlenecks[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2018: 4510-4520.
|
| [13] |
ZHANG Y, TIAN Y, KONG Y, et al. Residual dense network for image super-resolution[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2018: 2472-2481.
|
| [14] |
SHI W, Caballero J, Huszár F, et al. Real-time single image and video super-resolution using an efficient sub-pixel convolutional neural network[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2016: 1874-1883.
|
| [15] |
QIN X, ZHANG Z, HUANG C, et al. U2-Net: Going deeper with nested U-structure for salient object detection[J]. Pattern Recognition, 2020, 106: 107404. DOI: 10.1016/j.patcog.2020.107404
|
| [16] |
MA J, XU H, JIANG J, et al. DDcGAN: A dual-discriminator conditional generative adversarial network for multi-resolution image fusion[J]. IEEE Transactions on Image Processing, 2020, 29: 4980-4995. DOI: 10.1109/TIP.2020.2977573
|
| [17] |
LI H, WU X J. DenseFuse: A fusion approach to infrared and visible images[J]. IEEE Transactions on Image Processing, 2018, 28(5): 2614-2623.
|
| [18] |
LI H, XU T, WU X J, et al. Lrrnet: A novel representation learning guided fusion network for infrared and visible images[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2023, 45(9): 11040-11052. DOI: 10.1109/TPAMI.2023.3268209
|
| [19] |
LI H, WU X J, Durrani T. NestFuse: An infrared and visible image fusion architecture based on nest connection and spatial/channel attention models[J]. IEEE Transactions on Instrumentation and Measurement, 2020, 69(12): 9645-9656. DOI: 10.1109/TIM.2020.3005230
|
| [20] |
LI H, WU X J, Kittler J. RFN-Nest: An end-to-end residual fusion network for infrared and visible images[J]. Information Fusion, 2021, 73: 72-86. DOI: 10.1016/j.inffus.2021.02.023
|
| [21] |
TANG L, YUAN J, MA J. Image fusion in the loop of high-level vision tasks: A semantic-aware real-time infrared and visible image fusion network[J]. Information Fusion, 2022, 82: 28-42. DOI: 10.1016/j.inffus.2021.12.004
|
| [1] | LIAO Guangfeng, GUAN Zhiwei, CHEN Qiang. An Improved Dual Discriminator Generative Adversarial Network Algorithm for Infrared and Visible Image Fusion[J]. Infrared Technology , 2025, 47(3): 367-375. |
| [2] | DAI Yueming, YANG Lufeng, TONG Xiongmin. Real-time Section State Verification Method of Energy Management System Low Voltage Equipment Based on Infrared Image and Deep Learning[J]. Infrared Technology , 2024, 46(12): 1464-1470. |
| [3] | CHEN Haipeng, JIN Weiqi, LI Li, QIU Su, YU Xiangzhi. Study on BRDF Scattering Characteristics of Relay Wall in Non-Line-of-Sight Imaging Based on Time-gated SPAD Array[J]. Infrared Technology , 2024, 46(11): 1225-1234. |
| [4] | ZHONG Guoli, LIAO Shouyi, YANG Xinjie. Real-Time Infrared Image Generation of Battlefield Environment Based on JRM[J]. Infrared Technology , 2024, 46(2): 183-189. |
| [5] | SHEN Ji, NA Qiyue, XU Jiandong, CHANG Weijing, ZHANG Wei, JIAN Yunfei. 640×512 Frame Transfer EMCCD Camera Timing Sequence Design[J]. Infrared Technology , 2023, 45(5): 548-552. |
| [6] | WANG Mingxing, ZHENG Fu, WANG Yanqiu, SUN Zhibin. Time-of-Flight Point Cloud Denoising Method Based on Confidence Level[J]. Infrared Technology , 2022, 44(5): 513-520. |
| [7] | LIU Zhaoqing, LI Li, DONG Bing, JIN Weiqi. Shack-Hartman Detector Real-time Wavefront Processor Based on FPGA[J]. Infrared Technology , 2021, 43(8): 717-722. |
| [8] | CHEN Zheng, FU Kuisheng, DING Haishan. Analysis of the Influence of Installation Errors of an Infrared Stabilized Platform on Line-of-sight Angular Velocity[J]. Infrared Technology , 2021, 43(2): 110-115. |
| [9] | WEI Jiali, QU Huidong, WANG Yongxian, ZHU Junqing, GUAN Yingjun. Research Review of 3D Cameras Based on Time-of-Flight Method[J]. Infrared Technology , 2021, 43(1): 60-67. |
| [10] | HUANG Minshuang, GUAN Zaihui, JIANG Bo. Pulse Laser Ranging Using Sinusoidal Amplitude Time Conversion[J]. Infrared Technology , 2020, 42(5): 483-487. |
Supported by: Beijing Renhe Information Technology Co., Ltd. 
DownLoad: