Infrared and Visible Image Fusion Based on Multi-scale and Attention Model

HUANG Linglin; LI Qiang; LU Jinzheng; HE Xianzhen; PENG Bo

Volume 45 Issue 2

Feb. 2023

Turn off MathJax

Article Contents

Abstract

References

Infrared Technology > 2023 > 45(2): 143-149.

HUANG Linglin, LI Qiang, LU Jinzheng, HE Xianzhen, PENG Bo. Infrared and Visible Image Fusion Based on Multi-scale and Attention Model[J]. Infrared Technology , 2023, 45(2): 143-149.

Citation:

HUANG Linglin, LI Qiang, LU Jinzheng, HE Xianzhen, PENG Bo. Infrared and Visible Image Fusion Based on Multi-scale and Attention Model[J]. Infrared Technology , 2023, 45(2): 143-149.

Citation:

HUANG Linglin, LI Qiang, LU Jinzheng, HE Xianzhen, PENG Bo. Infrared and Visible Image Fusion Based on Multi-scale and Attention Model[J]. Infrared Technology , 2023, 45(2): 143-149.

PDF (979 KB)

Infrared and Visible Image Fusion Based on Multi-scale and Attention Model

HUANG Linglin^1,,
LI Qiang^{1, 2, ,},
LU Jinzheng¹,
HE Xianzhen¹,
PENG Bo^{1, 2}

1.
School of Information Engineering, Southwest University of Science and Technology, Mianyang 621010, China
2.
Robot Technology Used for Special Environment Key Laboratory of Sichuan Province, Southwest University of Science and Technology, Mianyang 621010, China

More Information

Received Date: February 24, 2021
Revised Date: April 11, 2021

Graphical Abstract

Abstract

Abstract

Aiming at the problems that infrared and visible images are prone to artifacts and unclear outlines of small targets after fusion, an infrared and visible images fusion algorithm based on the combination of multi-scale features and attention model is proposed. The feature maps of different scales of the source image are extracted through five times of down-sampling, and then the infrared and visible image feature maps of the same scale are input to the fusion layer based on the attention model to obtain an enhanced fusion feature map. Finally, the small-scale fusion feature map is up-sampled five times, and then added to the feature map of the same scale after up-sampling, until the scale is consistent with the source image, and the multi-scale fusion of the feature map is realized. Experiments compare the entropy, standard deviation, mutual information, edge retention, wavelet feature mutual information, visual information fidelity, and fusion efficiency of fused images under different fusion frameworks. The method in this paper is superior to the comparison algorithm in most indicators, and the target details are obvious and the outline are clear in the fused images.
- image fusion,
- multi-scale feature fusion,
- attention model,
- infrared images

FullText(HTML)

References (17)

References

[1]	赵立昌, 张宝辉, 吴杰, 等. 基于灰度能量差异性的红外与可见光图像融合[J]. 红外技术, 2020, 42(8): 775-782. https://www.cnki.com.cn/Article/CJFDTOTAL-HWJS202008012.htm ZHAO Lichang, ZHANG Baohui, WU Jie, et al. Fusion of infrared and visible images based on gray energy difference[J]. Infrared Technology, 2020, 42(8): 775-782. https://www.cnki.com.cn/Article/CJFDTOTAL-HWJS202008012.htm
[2]	白玉, 侯志强, 刘晓义, 等. 基于可见光图像和红外图像决策级融合的目标检测算法[J]. 空军工程大学学报(自然科学版), 2020, 21(6): 53-59, 100. DOI: 10.3969/j.issn.1009-3516.2020.06.009 BAI Yu, HOU Zhiqiang, LIU Xiaoyi, et al. An object detection algorithm based on decision-level fusion of visible light image and infrared image[J]. Journal of Air Force Engineering University(Natural Science Editon), 2020, 21(6): 53-59, 100. DOI: 10.3969/j.issn.1009-3516.2020.06.009
[3]	董安勇, 杜庆治, 苏斌, 等. 基于卷积神经网络的红外与可见光图像融合[J]. 红外技术, 2020, 42(7): 660-669. http://hwjs.nvir.cn/article/id/hwjs202007009 DONG Anyong, DU Qingzhi, SU Bin, et al. Infrared and visible image fusion based on convolutional neural network[J]. Infrared Technology, 2020, 42(7): 660-669. http://hwjs.nvir.cn/article/id/hwjs202007009
[4]	陈卓, 方明, 柴旭, 等. 红外与可见光图像融合的U-GAN模型[J]. 西北工业大学学报, 2020, 38(4): 904-912. DOI: 10.3969/j.issn.1000-2758.2020.04.027 CHEN Zhuo, FANG Ming, CHAI Xu, et al. Infrared and visible image fusion of U-GAN model[J]. Journal of Northwestern Polytechnical University, 2020, 38(4): 904-912. DOI: 10.3969/j.issn.1000-2758.2020.04.027
[5]	陈潮起, 孟祥超, 邵枫, 等. 一种基于多尺度低秩分解的红外与可见光图像融合方法[J]. 光学学报, 2020, 40(11): 72-80. https://www.cnki.com.cn/Article/CJFDTOTAL-GXXB202011008.htm CHEN Chaoqi, MENG Xiangchao, SHAO Feng, et al. Infrared and visible image fusion method based on multiscale low-rank decomposition [J]. Acta Optica Sinica, 2020, 40(11): 72-80. https://www.cnki.com.cn/Article/CJFDTOTAL-GXXB202011008.htm
[6]	林子慧. 基于多尺度变换的红外与可见光图像融合技术研究[D]. 成都: 中国科学院大学(中国科学院光电技术研究所), 2019. LIN Zihui. Research on Infrared and Visible Image Fusion Based on Multi-scale Trandform[D]. Chengdu: The Chinese Academy of Sciences(The Institute of Optics and Electronics), 2019.
[7]	马旗, 朱斌, 张宏伟. 基于VGG网络的双波段图像融合方法[J]. 激光与红外, 2019, 49(11): 1374-1380. DOI: 10.3969/j.issn.1001-5078.2019.11.018 MA Qi, ZHU Bin, ZHANG Hongwei. Dual-band image fusion method based on VGGNet[J]. Laser & Infrared, 2019, 49(11): 1374-1380. DOI: 10.3969/j.issn.1001-5078.2019.11.018
[8]	LI H, WU X, Durrani T S. Infrared and visible image fusion with ResNet and zero-phase component analysis[J]. Infrared Physics & Technology, 2019, 102: 103039.
[9]	LIN T, Dollár P, Girshick R, et al. Feature pyramid networks for object detection[C]//2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2017: 936-944.
[10]	Woo S, Park J, Lee J, et al. CBAM: Convolution-al Block Attention Module[C]//ECCV, 2018: 3-19.
[11]	LI H, WU X J, Durrani T. NestFuse: an infrared and visible image fusion architecture based on nest connection and spatial/channel attention models [J]. IEEE Transactions on Instrumentation and Measurement, 2020, 12(69): 9645-9656.
[12]	杨艳春, 李娇, 王阳萍. 图像融合质量评价方法研究综述[J]. 计算机科学与探索, 2018, 12(7): 1021-1035. https://www.cnki.com.cn/Article/CJFDTOTAL-KXTS201807002.htm YANG Yanchun, LI Jiao, WANG Yangping. Review of image fusion quality evaluation methods[J]. Journal of Frontiers of Computer Science and Technology, 2018, 12(7): 1021-1035. https://www.cnki.com.cn/Article/CJFDTOTAL-KXTS201807002.htm
[13]	LIN T Y, Maire M, Belongie S, et al. Microsoft coco: common objects in context[C]//ECCV, 2014: 3-5.
[14]	Toet A. TNO Image Fusion Dataset. figshare. Dataset[DB/OL]. https://doi.org/10.6084/m9.figshare.1008029.v2, 2014.
[15]	LI H, WU X J. DenseFuse: a fusion approach to infrared and visible images[J]. IEEE Trans. Image Process, 2019, 28(5): 2614-2623.
[16]	Prabhakar K R, Srikar V S, Babu R V. DeepFuse: a deep unsuper-vised approach for exposure fusion with extreme exposure image pairs[C]//2017 IEEE International Conference on Computer Vision (ICCV), 2017: 4724-4732.
[17]	MA J, ZHOU Z, WANG B. et al. Infrared and visible image fusion based on visual saliency map and weighted least square optimization[J]. Infrared Physics & Technology, 2017, 82: 8-17.

[1]	LI Jianghui. A Method and System for Infrared Image Simulation Based on ModelSim[J]. Infrared Technology , 2024, 46(7): 802-806.
[2]	WANG Xia, ZHAO Jiabi, SUN Qiyang, JIN Weiqi. Performance Evaluation Model for Infrared Polarization Imaging System[J]. Infrared Technology , 2023, 45(5): 437-445.
[3]	KONG Derui, XIA Ming, LI Haiying, CHEN Jun, ZHAO Peng. Theoretical Analysis and Matlab Simulation of Dynamic Vibration Absorber for Single-Piston Linear Compressor[J]. Infrared Technology , 2021, 43(10): 1014-1021.
[4]	ZHANG Jingyang, YAN Limin, CHEN Zhiheng. Nighttime Fog Removal Using the Dark Point Light Source Model[J]. Infrared Technology , 2021, 43(8): 798-803.
[5]	HU Yang, CHEN Cheng, HUA Sangtun, QIU Yafeng. Thermal Calculation of Countercurrent Cooling Tower and Design of Infrared Thermal Image Temperature Control System[J]. Infrared Technology , 2021, 43(3): 225-229.
[6]	PAN Hao, MA Yi, ZHOU Fangrong, MA Yutang, QIAN Guochao, WEN Gang. Research on the Theoretical Model Between Solar-blind UV and Atmospheric Temperature during Atmospheric Transmission[J]. Infrared Technology , 2020, 42(10): 1007-1012.
[7]	HAN Kun, YAO Ze, QIAO Kai, YANG Shuning, HE Yingping. Theoretical Model of Dynamic MTF of Low-Light-Level ICCD[J]. Infrared Technology , 2020, 42(3): 294-299.
[8]	SUN Jianning, SI Shuguang, WANG Xingchao, JIN Muchun, LI Dong, REN Ling, HOU Wei, ZHAO Min, GU Ying, QIAO Fangjian, ZHANG Haoda, CAO Yiqi. Preparation Method of K2CsSb Photocathode Using the Reflectance Theory Model[J]. Infrared Technology , 2017, 39(12): 1087-1091.
[9]	ZHANG Yao-jun, WU Gui-ling, LI Lei. Fusion for Infrared and Visible Light Images Based on Shearlet Transform and Quantum Theory Model[J]. Infrared Technology , 2015, (5): 418-423.
[10]	Theoretic Module of Uncooled IR Detector Performance Improvement[J]. Infrared Technology , 2002, 24(4): 31-34. DOI: 10.3969/j.issn.1001-8891.2002.04.009

Cited By

Cited by

Periodical cited type(1)

邱祥彪，杨晓明，孙建宁，王健，丛晓庆，金戈，曾进能，张正君，潘凯，陈晓倩. 高空间分辨微通道板现状及发展. 红外技术. 2024(04): 460-466 .

本站查看

Other cited types(0)

Get Citation

PDF

XML

Article views (170) PDF downloads (57) Cited by(1)

Infrared and Visible Image Fusion Based on Multi-scale and Attention Model

Abstract

References

Related Articles

Cited by

Periodical cited type(1)

Other cited types(0)

Catalog

Related

Infrared and Visible Image Fusion Based on Multi-scale and Attention Model

Abstract

References

Related Articles

Cited by

Periodical cited type(1)

Other cited types(0)

Catalog

Related

Export File

Citation

Format

Content