基于FCM与引导滤波的红外与可见光图像融合

蒋杰伟; 刘尚辉; 金库; 刘海洋; 魏戌盟; 巩稼民

基于FCM与引导滤波的红外与可见光图像融合

1.
西安邮电大学电子工程学院, 陕西西安 710121
2.
西安邮电大学通信与信息工程学院, 陕西西安 710121

基金项目:

国家自然科学基金 61775180

国家自然科学基金 62276210

陕西省自然科学基础研究计划 2022JM-380

详细信息

作者简介:
蒋杰伟（1982-），男，河南商丘人，讲师，博士，主要从事人工智能、机器学习方面的研究

通讯作者:
刘尚辉（1996-），男，陕西商洛人，硕士研究生，主要从事红外光与可见光图像融合方面的研究。E-mail: lsh81687039@163.com

中图分类号: TP391
计量
- 文章访问数: 142
- HTML全文浏览量: 38
- PDF下载量: 35
- 被引次数: 0
出版历程
- 收稿日期: 2022-08-16
- 修回日期: 2022-09-13
- 刊出日期: 2023-03-20

Infrared and Visible-Light Image Fusion Based on FCM and Guided Filtering

1.
School of Electronic Engineering, Xi'an University of Posts and Telecommunications, Xi 'an 710121, China
2.
School of Communication and Information Engineering, Xi 'an University of Posts and Telecommunications, Xi 'an 710121, China

摘要

摘要: 针对传统红外与可见光图像融合算法中存在的目标模糊、细节丢失、算法不稳定等问题，提出了一种基于模糊C均值聚类（Fuzzy C-means, FCM）与引导滤波的红外与可见光图像融合方法。原图像经过非下采样剪切波变换（Nonsubsampled Shearlet Transform, NSST）后对低频子带进行引导滤波增强，再利用FCM与双通道脉冲发放皮层模型（Dual Channel Spiking Cortical Model, DCSCM）结合对高低频子带进行融合，最后经NSST逆变换得到融合图像。实验结果表明，本文算法稳定，主观评价上所得融合图像目标明确，细节保留较为完整，客观评价上在标准差、互信息、平均梯度、信息熵和边缘保留因子等评价标准中表现优良。
- 图像处理 /
- 模糊C均值聚类 /
- 引导滤波 /
- 双通道脉冲发放皮层模型
Abstract: To solve the problems of vague targets, detail loss, and algorithm instability in traditional infrared and visible-light image fusion algorithms, a fusion method based on fuzzy c-means (FCM) clustering and guided filtering is proposed. The low-frequency sub-band was enhanced by guided filtering after applying a non-subsampled shearlet transform (NSST) to the original image. The low- and high-frequency sub-bands were then fused using FCM clustering and a dual-channel spiking cortical model. Finally, the fused image was obtained using an inverse NSST transform. The experimental results showed that the proposed algorithm was stable, the fusion image had clear targets and relatively complete details in the subjective evaluation, and the algorithm had an excellent standard deviation, mutual information, average gradient, information entropy, and edge retention factor in the objective evaluation.
- image processing /
- fuzzy C-mean /
- guided filtering /
- dual channel spiking cortical model

HTML全文

图 1 双通道脉冲发放皮层模型

Figure 1. Dual channel spiking cortical model

下载: 全尺寸图片幻灯片

图 2 本文融合方法

Figure 2. The proposed fusion method

下载: 全尺寸图片幻灯片

图 3 第1组图像融合结果。(a) 第1组源可见光图像; (b) 第1组源红外图像; (c) MGFF; (d) MSD; (e) MTD; (f) VIP; (g) FCMA; (h) 本文方法

Figure 3. Image fusion results of the first group. (a) Visible image of set 1; (b) infrared image of set 1; (c) MGFF; (d) MSD; (e) MTD; (f) VIP; (g) FCMA; (h) Proposed method

下载: 全尺寸图片幻灯片

图 4 第2组图像融合结果。(a) 第2组源可见光图像; (b) 第2组源红外图像; (c) MGFF; (d) MSD; (e) MTD; (f) VIP; (g) FCMA; (h) 本文方法

Figure 4. Image fusion results of the second group. (a) Visible image of set 2; (b) infrared image of set 2; (c) MGFF; (d) MSD; (e) MTD; (f) VIP; (g) FCMA; (h) Proposed method

下载: 全尺寸图片幻灯片

图 5 第3组图像融合结果。(a) 第3组源可见光图像; (b) 第3组源红外图像; (c) MGFF; (d) MSD; (e) MTD; (f) VIP; (g) FCMA; (h) 本文方法

Figure 5. Image fusion results of the third group. (a) Visible image of set 3; (b) infrared image of set 3; (c) MGFF; (d) MSD; (e) MTD; (f) VIP; (g) FCMA; (h) Proposed method

下载: 全尺寸图片幻灯片

表 1 主观评价尺度评分

Table 1. Subjective evaluation scale score table

Score	Quality scale	Obstruction scale
5	very nice	Lossless image quality
4	nice	The image quality is damaged, but it does not hinder viewing
3	normal	Clearly see that the image quality is damaged
2	poor	Obstruction to viewing
1	very poor	Serious impact on viewing

下载: 导出CSV

表 2 五分制评价结果

Table 2. Five point evaluation results

	First set of image scores	Second set of image scores	Third set of image scores
Professional person 1	4	5	4
Professional person 2	5	5	4
Professional person 3	4	4	4
Nonprofessional person 1	5	5	5
Nonprofessional person 2	5	5	4
Average score	4.6	4.8	4.2

下载: 导出CSV

表 3 客观评价指标

Table 3. Objective evaluation results

Image	Algorithm	STD	MI	AG	EN	Q^AB/F	SSIM
Group 1	MGFF	48.5438	2.5642	10.7398	7.3355	0.5691	0.5038
	MSD	48.3475	2.5589	11.2680	7.2762	0.5925	0.4877
	MTD	43.3842	3.0839	9.8755	6.9701	0.5456	0.4722
	VIP	44.8607	0.5109	10.4776	7.2307	0.5665	0.6142
	FCMA	43.9961	3.1582	10.5662	7.3527	0.6230	0.4964
	Proposed	45.0086	3.1720	10.7624	7.3768	0.5978	0.5090
Group 2	MGFF	36.6809	1.7426	4.9351	6.8599	0.4702	0.5268
	MSD	52.3717	2.5234	4.7900	7.0811	0.4706	0.4854
	MTD	52.1024	3.0416	4.3414	6.8654	0.4563	0.4920
	VIP	52.8195	0.3818	4.3009	6.9521	0.5332	0.7334
	FCMA	60.5238	3.1647	4.6397	7.3857	0.4877	0.4375
	Proposed	60.1718	3.2102	4.6594	7.4388	0.4564	0.4693
Group 3	MGFF	40.0211	1.5924	6.7958	7.2387	0.4799	0.5095
	MSD	49.7948	2.3439	6.8837	7.2386	0.5371	0.4871
	MTD	60.7380	4.4287	6.3965	7.1101	0.5810	0.4641
	VIP	56.0103	0.6042	5.6657	6.7389	0.5663	0.6618
	FCMA	57.0775	2.3680	6.1483	7.2681	0.4667	0.4515
	Proposed	57.4021	3.0526	6.6048	7.2777	0.5556	0.4753

下载: 导出CSV

参考文献(20)

[1]	LI Shutao, KANG Xudong, FANG Leyuan, et al. Pixel-level image fusion: a survey of the state of the art[J]. Information Fusion, 2017, 33(1): 100-112
[2]	蔡李美, 李新福, 田学东. 基于分层图像融合的虚拟视点绘制算法[J]. 计算机工程, 2021, 47(4): 204-210. DOI: 10.19678/j.issn.1000-3428.0058057. CAI L M, LI X F, TIAN X D. Virtual viewpoint rendering algorithm based on hierarchical image fusio [J]. Computer Engineering, 2021, 47(4): 204-210. DOI: 10.19678/j.issn.1000-3428.0058057
[3]	冯鑫, 张建华, 胡开群, 等. 基于变分多尺度的红外与可见光图像融合[J]. 电子学报, 2018, 46(3): 680-687. https://www.cnki.com.cn/Article/CJFDTOTAL-DZXU201803025.htm FENG X, ZHANG J H, HU K Q, et al. The infrared and visible image fusion method based on variational multiscale[J]. Acta Electronica Sinica, 2018, 46(3): 680-687. https://www.cnki.com.cn/Article/CJFDTOTAL-DZXU201803025.htm
[4]	LIU Y, LIU S, WANG Z. A general framework for image fusion based on multi-scale transform and sparse representation[J]. Information Fusion, 2015, 24: 147-164. doi: 10.1016/j.inffus.2014.09.004
[5]	李威, 李忠民. NSST域红外和可见光图像感知融合[J]. 激光与光电子学进展, 2021, 58(20): 202-210. https://www.cnki.com.cn/Article/CJFDTOTAL-JGDJ202120021.htm LI W, LI Z M. NSST-Based perception fusion method for infrared and visible images[J]. Laser & Optoelectronics Progress, 2021, 58(20): 202-210. https://www.cnki.com.cn/Article/CJFDTOTAL-JGDJ202120021.htm
[6]	ZHAO W D, LU H C. Medical image fusion and denoising with altering sequential filter and adaptive fractional order total variation[J]. IEEE Transactions on Instrumentation and Measurement, 2017, 66(9) : 2283-2294 doi: 10.1109/TIM.2017.2700198
[7]	任亚飞, 张娟梅. 基于NSST多尺度熵的红外与可见光图像融合[J]. 兵器装备工程学报, 2022, 43(7): 278-285. https://www.cnki.com.cn/Article/CJFDTOTAL-CUXI202207042.htm REN Y F, ZHANG J M. Infrared and visible image fusion based on NSST multi-scale entropy[J]. Journal of Ordnance Equipment Engineering, 2022, 43(7): 278-285. https://www.cnki.com.cn/Article/CJFDTOTAL-CUXI202207042.htm
[8]	ZHOU Z Q, WANG B, LI S, et al. Perceptual fusion of infrared and visible images through a hybrid multiscale decomposition with Gaussian and bilateral filters[J]. Information Fusion, 2016, 30(30): 15-26.
[9]	李文, 叶坤涛, 舒蕾蕾, 等. 基于高斯模糊逻辑和ADCSCM的红外与可见光图像融合算法[J]. 红外技术, 2022, 44(7): 693-701. http://hwjs.nvir.cn/article/id/227ae3cd-57b4-4ec7-a248-bdc1de60993c LI W, YE K T, SHU L L, et al. Infrared and visible image fusion algorithm based on gaussian fuzzy logic and adaptive dual-channel spiking cortical model[J]. Infrared Technology, 2022, 44(7): 693-701. http://hwjs.nvir.cn/article/id/227ae3cd-57b4-4ec7-a248-bdc1de60993c
[10]	张莲, 杨森淋, 禹红良, 等. 改进非局部核模糊C-均值聚类的红外图像分割[J]. 重庆理工大学学报: 自然科学, 2020, 34(11): 130-137. https://www.cnki.com.cn/Article/CJFDTOTAL-CGGL202011018.htm ZHANG L, YANG S L, YU H L, et al. Improved infrared image segmentation based on nonlocal nuclear fuzzy C-means clustering[J]. Journal of Chongqing University of Technology: Natural Science, 2020, 34(11): 130-137. https://www.cnki.com.cn/Article/CJFDTOTAL-CGGL202011018.htm
[11]	赵程, 黄永东. 基于滚动导向滤波和混合多尺度分解的红外与可见光图像融合方法[J]. 激光与光电子学进展, 2019, 56(14): 106-120. https://www.cnki.com.cn/Article/CJFDTOTAL-JGDJ201914013.htm ZHAO C, HUANG Y D. Infrared and visible image fusion via rolling guidance filtering and hybrid multi-sacle decomposition[J]. Laser & Optoelectronics Progress, 2019, 56(14): 106-120. https://www.cnki.com.cn/Article/CJFDTOTAL-JGDJ201914013.htm
[12]	李旭超, 刘海宽, 王飞, 等. 图像分割中的模糊聚类方法[J]. 中国图象图形学报, 2012, 17(4): 447-458. https://www.cnki.com.cn/Article/CJFDTOTAL-ZGTB201204004.htm LI X C, LIU H K, WANG F, et al. The survey of fuzzy clustering method for image segmentation[J]. Journal of Image and Graphics, 2012, 17(4): 447-458. https://www.cnki.com.cn/Article/CJFDTOTAL-ZGTB201204004.htm
[13]	谢伟, 周玉钦, 游敏. 融合梯度信息的改进引导滤波[J]. 中国图象图形学报, 2016, 21(9): 1119-1126. https://www.cnki.com.cn/Article/CJFDTOTAL-ZGTB201609001.htm XIE W, ZHOU Y, YOU M. Improved guided image filtering integrated with gradient information[J]. Journal of Image and Graphics, 2016, 21(9): 1119-1126. https://www.cnki.com.cn/Article/CJFDTOTAL-ZGTB201609001.htm
[14]	江泽涛, 吴辉, 周哓玲. 基于改进引导滤波和双通道脉冲发放皮层模型的红外与可见光图像融合算法[J]. 光学学报, 2018, 38(2): 0210002. https://www.cnki.com.cn/Article/CJFDTOTAL-GXXB201802015.htm JIANG Z T, WU H, ZHOU X L. Infrared and visible image fusion algorithm based on improved guided filtering and dual-channel spiking cortical model[J]. Acta Optica Sinica, 2018, 38(2): 0210002. https://www.cnki.com.cn/Article/CJFDTOTAL-GXXB201802015.htm
[15]	HUANG W, JING Z L. Evaluation of focus measures in multi-foucus image fusion[J]. Pattern Recognition Letters, 2007, 28(4): 493-500
[16]	Bavirisetti D P, XIAO G, ZHAO J, et al. Multi-scale guided image and video fusion: a fast and efficient approach[J]. Circuits Syst Signal Process, 2019, 38: 5576–5605.
[17]	ZHOU Z Q, WANG B, LI S, et al. Perceptual fusion of infrared and visible images through a hybrid multi-scale decomposition with Gaussian and bilateral filters[J]. Information Fusion, 2016, 30: 16-25.
[18]	CHEN J, LI X J, LUO L B, et al. Infrared and visible image fusion based on target-enhanced multiscale transform decomposition[J]. Information Sciences, 2020, 508: 64-78.
[19]	ZHANG Y, ZHANG L J, BAI X Z, et al. Infrared and visual image fusion through infrared feature extraction and visual information preservation[J]. Infrared Physics and Technology, 2017, 83: 227-237.
[20]	巩稼民, 刘爱萍, 张晨, 等. 基于FCM与ADSCM的红外与可见光图像融合[J]. 激光与光电子学进展, 2020, 57(20): 222-230. https://www.cnki.com.cn/Article/CJFDTOTAL-JGDJ202020025.htm GONG J M, LIU A P, ZHANG C, et al. Infrared and visible light image fusion based on FCM and ADSCM[J]. Laser & Optoelectronics Progress, 2020, 57(20): 222-230. https://www.cnki.com.cn/Article/CJFDTOTAL-JGDJ202020025.htm