基于FCM与引导滤波的红外与可见光图像融合

蒋杰伟; 刘尚辉; 金库; 刘海洋; 魏戌盟; 巩稼民

基于FCM与引导滤波的红外与可见光图像融合

1.
西安邮电大学电子工程学院, 陕西西安 710121
2.
西安邮电大学通信与信息工程学院, 陕西西安 710121

基金项目:

国家自然科学基金 61775180

国家自然科学基金 62276210

陕西省自然科学基础研究计划 2022JM-380

详细信息

作者简介:
蒋杰伟（1982-），男，河南商丘人，讲师，博士，主要从事人工智能、机器学习方面的研究

通讯作者:
刘尚辉（1996-），男，陕西商洛人，硕士研究生，主要从事红外光与可见光图像融合方面的研究。E-mail: lsh81687039@163.com

中图分类号: TP391
计量
- 文章访问数: 176
- HTML全文浏览量: 45
- PDF下载量: 41
出版历程
- 收稿日期: 2022-08-15
- 修回日期: 2022-09-12
- 刊出日期: 2023-03-19

Infrared and Visible-Light Image Fusion Based on FCM and Guided Filtering

1.
School of Electronic Engineering, Xi'an University of Posts and Telecommunications, Xi 'an 710121, China
2.
School of Communication and Information Engineering, Xi 'an University of Posts and Telecommunications, Xi 'an 710121, China

摘要

摘要: 针对传统红外与可见光图像融合算法中存在的目标模糊、细节丢失、算法不稳定等问题，提出了一种基于模糊C均值聚类（Fuzzy C-means, FCM）与引导滤波的红外与可见光图像融合方法。原图像经过非下采样剪切波变换（Nonsubsampled Shearlet Transform, NSST）后对低频子带进行引导滤波增强，再利用FCM与双通道脉冲发放皮层模型（Dual Channel Spiking Cortical Model, DCSCM）结合对高低频子带进行融合，最后经NSST逆变换得到融合图像。实验结果表明，本文算法稳定，主观评价上所得融合图像目标明确，细节保留较为完整，客观评价上在标准差、互信息、平均梯度、信息熵和边缘保留因子等评价标准中表现优良。
- 图像处理 /
- 模糊C均值聚类 /
- 引导滤波 /
- 双通道脉冲发放皮层模型
Abstract: To solve the problems of vague targets, detail loss, and algorithm instability in traditional infrared and visible-light image fusion algorithms, a fusion method based on fuzzy c-means (FCM) clustering and guided filtering is proposed. The low-frequency sub-band was enhanced by guided filtering after applying a non-subsampled shearlet transform (NSST) to the original image. The low- and high-frequency sub-bands were then fused using FCM clustering and a dual-channel spiking cortical model. Finally, the fused image was obtained using an inverse NSST transform. The experimental results showed that the proposed algorithm was stable, the fusion image had clear targets and relatively complete details in the subjective evaluation, and the algorithm had an excellent standard deviation, mutual information, average gradient, information entropy, and edge retention factor in the objective evaluation.
- image processing /
- fuzzy C-mean /
- guided filtering /
- dual channel spiking cortical model

HTML全文

图 1 双通道脉冲发放皮层模型

Figure 1. Dual channel spiking cortical model

下载: 全尺寸图片幻灯片

图 2 本文融合方法

Figure 2. The proposed fusion method

下载: 全尺寸图片幻灯片

图 3 第1组图像融合结果。(a) 第1组源可见光图像; (b) 第1组源红外图像; (c) MGFF; (d) MSD; (e) MTD; (f) VIP; (g) FCMA; (h) 本文方法

Figure 3. Image fusion results of the first group. (a) Visible image of set 1; (b) infrared image of set 1; (c) MGFF; (d) MSD; (e) MTD; (f) VIP; (g) FCMA; (h) Proposed method

下载: 全尺寸图片幻灯片

图 4 第2组图像融合结果。(a) 第2组源可见光图像; (b) 第2组源红外图像; (c) MGFF; (d) MSD; (e) MTD; (f) VIP; (g) FCMA; (h) 本文方法

Figure 4. Image fusion results of the second group. (a) Visible image of set 2; (b) infrared image of set 2; (c) MGFF; (d) MSD; (e) MTD; (f) VIP; (g) FCMA; (h) Proposed method

下载: 全尺寸图片幻灯片

图 5 第3组图像融合结果。(a) 第3组源可见光图像; (b) 第3组源红外图像; (c) MGFF; (d) MSD; (e) MTD; (f) VIP; (g) FCMA; (h) 本文方法

Figure 5. Image fusion results of the third group. (a) Visible image of set 3; (b) infrared image of set 3; (c) MGFF; (d) MSD; (e) MTD; (f) VIP; (g) FCMA; (h) Proposed method

下载: 全尺寸图片幻灯片

表 1 主观评价尺度评分

Table 1 Subjective evaluation scale score table

Score	Quality scale	Obstruction scale
5	very nice	Lossless image quality
4	nice	The image quality is damaged, but it does not hinder viewing
3	normal	Clearly see that the image quality is damaged
2	poor	Obstruction to viewing
1	very poor	Serious impact on viewing

下载: 导出CSV

表 2 五分制评价结果

Table 2 Five point evaluation results

	First set of image scores	Second set of image scores	Third set of image scores
Professional person 1	4	5	4
Professional person 2	5	5	4
Professional person 3	4	4	4
Nonprofessional person 1	5	5	5
Nonprofessional person 2	5	5	4
Average score	4.6	4.8	4.2

下载: 导出CSV

表 3 客观评价指标

Table 3 Objective evaluation results

Image	Algorithm	STD	MI	AG	EN	Q^AB/F	SSIM
Group 1	MGFF	48.5438	2.5642	10.7398	7.3355	0.5691	0.5038
	MSD	48.3475	2.5589	11.2680	7.2762	0.5925	0.4877
	MTD	43.3842	3.0839	9.8755	6.9701	0.5456	0.4722
	VIP	44.8607	0.5109	10.4776	7.2307	0.5665	0.6142
	FCMA	43.9961	3.1582	10.5662	7.3527	0.6230	0.4964
	Proposed	45.0086	3.1720	10.7624	7.3768	0.5978	0.5090
Group 2	MGFF	36.6809	1.7426	4.9351	6.8599	0.4702	0.5268
	MSD	52.3717	2.5234	4.7900	7.0811	0.4706	0.4854
	MTD	52.1024	3.0416	4.3414	6.8654	0.4563	0.4920
	VIP	52.8195	0.3818	4.3009	6.9521	0.5332	0.7334
	FCMA	60.5238	3.1647	4.6397	7.3857	0.4877	0.4375
	Proposed	60.1718	3.2102	4.6594	7.4388	0.4564	0.4693
Group 3	MGFF	40.0211	1.5924	6.7958	7.2387	0.4799	0.5095
	MSD	49.7948	2.3439	6.8837	7.2386	0.5371	0.4871
	MTD	60.7380	4.4287	6.3965	7.1101	0.5810	0.4641
	VIP	56.0103	0.6042	5.6657	6.7389	0.5663	0.6618
	FCMA	57.0775	2.3680	6.1483	7.2681	0.4667	0.4515
	Proposed	57.4021	3.0526	6.6048	7.2777	0.5556	0.4753

下载: 导出CSV

参考文献(20)

[1]	LI Shutao, KANG Xudong, FANG Leyuan, et al. Pixel-level image fusion: a survey of the state of the art[J]. Information Fusion, 2017, 33(1): 100-112
[2]	蔡李美, 李新福, 田学东. 基于分层图像融合的虚拟视点绘制算法[J]. 计算机工程, 2021, 47(4): 204-210. DOI: 10.19678/j.issn.1000-3428.0058057. CAI L M, LI X F, TIAN X D. Virtual viewpoint rendering algorithm based on hierarchical image fusio [J]. Computer Engineering, 2021, 47(4): 204-210. DOI: 10.19678/j.issn.1000-3428.0058057
[3]	冯鑫, 张建华, 胡开群, 等. 基于变分多尺度的红外与可见光图像融合[J]. 电子学报, 2018, 46(3): 680-687. https://www.cnki.com.cn/Article/CJFDTOTAL-DZXU201803025.htm FENG X, ZHANG J H, HU K Q, et al. The infrared and visible image fusion method based on variational multiscale[J]. Acta Electronica Sinica, 2018, 46(3): 680-687. https://www.cnki.com.cn/Article/CJFDTOTAL-DZXU201803025.htm
[4]	LIU Y, LIU S, WANG Z. A general framework for image fusion based on multi-scale transform and sparse representation[J]. Information Fusion, 2015, 24: 147-164. DOI: 10.1016/j.inffus.2014.09.004
[5]	李威, 李忠民. NSST域红外和可见光图像感知融合[J]. 激光与光电子学进展, 2021, 58(20): 202-210. https://www.cnki.com.cn/Article/CJFDTOTAL-JGDJ202120021.htm LI W, LI Z M. NSST-Based perception fusion method for infrared and visible images[J]. Laser & Optoelectronics Progress, 2021, 58(20): 202-210. https://www.cnki.com.cn/Article/CJFDTOTAL-JGDJ202120021.htm
[6]	ZHAO W D, LU H C. Medical image fusion and denoising with altering sequential filter and adaptive fractional order total variation[J]. IEEE Transactions on Instrumentation and Measurement, 2017, 66(9) : 2283-2294 DOI: 10.1109/TIM.2017.2700198
[7]	任亚飞, 张娟梅. 基于NSST多尺度熵的红外与可见光图像融合[J]. 兵器装备工程学报, 2022, 43(7): 278-285. https://www.cnki.com.cn/Article/CJFDTOTAL-CUXI202207042.htm REN Y F, ZHANG J M. Infrared and visible image fusion based on NSST multi-scale entropy[J]. Journal of Ordnance Equipment Engineering, 2022, 43(7): 278-285. https://www.cnki.com.cn/Article/CJFDTOTAL-CUXI202207042.htm
[8]	ZHOU Z Q, WANG B, LI S, et al. Perceptual fusion of infrared and visible images through a hybrid multiscale decomposition with Gaussian and bilateral filters[J]. Information Fusion, 2016, 30(30): 15-26.
[9]	李文, 叶坤涛, 舒蕾蕾, 等. 基于高斯模糊逻辑和ADCSCM的红外与可见光图像融合算法[J]. 红外技术, 2022, 44(7): 693-701. http://hwjs.nvir.cn/article/id/227ae3cd-57b4-4ec7-a248-bdc1de60993c LI W, YE K T, SHU L L, et al. Infrared and visible image fusion algorithm based on gaussian fuzzy logic and adaptive dual-channel spiking cortical model[J]. Infrared Technology, 2022, 44(7): 693-701. http://hwjs.nvir.cn/article/id/227ae3cd-57b4-4ec7-a248-bdc1de60993c
[10]	张莲, 杨森淋, 禹红良, 等. 改进非局部核模糊C-均值聚类的红外图像分割[J]. 重庆理工大学学报: 自然科学, 2020, 34(11): 130-137. https://www.cnki.com.cn/Article/CJFDTOTAL-CGGL202011018.htm ZHANG L, YANG S L, YU H L, et al. Improved infrared image segmentation based on nonlocal nuclear fuzzy C-means clustering[J]. Journal of Chongqing University of Technology: Natural Science, 2020, 34(11): 130-137. https://www.cnki.com.cn/Article/CJFDTOTAL-CGGL202011018.htm
[11]	赵程, 黄永东. 基于滚动导向滤波和混合多尺度分解的红外与可见光图像融合方法[J]. 激光与光电子学进展, 2019, 56(14): 106-120. https://www.cnki.com.cn/Article/CJFDTOTAL-JGDJ201914013.htm ZHAO C, HUANG Y D. Infrared and visible image fusion via rolling guidance filtering and hybrid multi-sacle decomposition[J]. Laser & Optoelectronics Progress, 2019, 56(14): 106-120. https://www.cnki.com.cn/Article/CJFDTOTAL-JGDJ201914013.htm
[12]	李旭超, 刘海宽, 王飞, 等. 图像分割中的模糊聚类方法[J]. 中国图象图形学报, 2012, 17(4): 447-458. https://www.cnki.com.cn/Article/CJFDTOTAL-ZGTB201204004.htm LI X C, LIU H K, WANG F, et al. The survey of fuzzy clustering method for image segmentation[J]. Journal of Image and Graphics, 2012, 17(4): 447-458. https://www.cnki.com.cn/Article/CJFDTOTAL-ZGTB201204004.htm
[13]	谢伟, 周玉钦, 游敏. 融合梯度信息的改进引导滤波[J]. 中国图象图形学报, 2016, 21(9): 1119-1126. https://www.cnki.com.cn/Article/CJFDTOTAL-ZGTB201609001.htm XIE W, ZHOU Y, YOU M. Improved guided image filtering integrated with gradient information[J]. Journal of Image and Graphics, 2016, 21(9): 1119-1126. https://www.cnki.com.cn/Article/CJFDTOTAL-ZGTB201609001.htm
[14]	江泽涛, 吴辉, 周哓玲. 基于改进引导滤波和双通道脉冲发放皮层模型的红外与可见光图像融合算法[J]. 光学学报, 2018, 38(2): 0210002. https://www.cnki.com.cn/Article/CJFDTOTAL-GXXB201802015.htm JIANG Z T, WU H, ZHOU X L. Infrared and visible image fusion algorithm based on improved guided filtering and dual-channel spiking cortical model[J]. Acta Optica Sinica, 2018, 38(2): 0210002. https://www.cnki.com.cn/Article/CJFDTOTAL-GXXB201802015.htm
[15]	HUANG W, JING Z L. Evaluation of focus measures in multi-foucus image fusion[J]. Pattern Recognition Letters, 2007, 28(4): 493-500
[16]	Bavirisetti D P, XIAO G, ZHAO J, et al. Multi-scale guided image and video fusion: a fast and efficient approach[J]. Circuits Syst Signal Process, 2019, 38: 5576–5605.
[17]	ZHOU Z Q, WANG B, LI S, et al. Perceptual fusion of infrared and visible images through a hybrid multi-scale decomposition with Gaussian and bilateral filters[J]. Information Fusion, 2016, 30: 16-25.
[18]	CHEN J, LI X J, LUO L B, et al. Infrared and visible image fusion based on target-enhanced multiscale transform decomposition[J]. Information Sciences, 2020, 508: 64-78.
[19]	ZHANG Y, ZHANG L J, BAI X Z, et al. Infrared and visual image fusion through infrared feature extraction and visual information preservation[J]. Infrared Physics and Technology, 2017, 83: 227-237.
[20]	巩稼民, 刘爱萍, 张晨, 等. 基于FCM与ADSCM的红外与可见光图像融合[J]. 激光与光电子学进展, 2020, 57(20): 222-230. https://www.cnki.com.cn/Article/CJFDTOTAL-JGDJ202020025.htm GONG J M, LIU A P, ZHANG C, et al. Infrared and visible light image fusion based on FCM and ADSCM[J]. Laser & Optoelectronics Progress, 2020, 57(20): 222-230. https://www.cnki.com.cn/Article/CJFDTOTAL-JGDJ202020025.htm

施引文献(17)

期刊类型引用(9)

1.	杨仁梅，赵艳，权军霞，方婷婷，白莎莎，费利燕. 1例血液透析患者股静脉导管周围医用粘胶剂相关性皮肤损伤的护理. 当代护士(中旬刊). 2025(04): 109-112 . 百度学术
2.	李猛，尚坤，陈树刚，刘秀斌，王奕霏，吴璠. 一种适用于载人航天飞行的针织手套设计及性能分析. 载人航天. 2024(01): 17-22 . 百度学术
3.	陈红，段小文，郭玲玲，范硕，祝成炎，张红霞. 远红外涤纶交织面料的开发及其结构性能. 上海纺织科技. 2024(04): 64-68 . 百度学术
4.	朱小英，朱丽舒，孔月明. 艾灸联合远红外线照射改善一例血液透析动静脉内瘘血肿的效果. 名医. 2024(11): 69-71 . 百度学术
5.	郑红菊，张方方，冯文艇. 低强度激光长期暴露对女性工人职业健康的影响. 职业卫生与应急救援. 2023(05): 595-598 . 百度学术
6.	侯刘林，李贺，宗珂. 等速肌力训练联合远红外线照射在乳腺癌根治术后患者中的应用效果. 癌症进展. 2022(10): 1024-1027 . 百度学术
7.	杨永健，李丽娟，庞永诚，杨海玲，龚瑞莹. 基于网络药理学探讨三黄紫参油治疗压力性损伤的作用机制. 湖南中医杂志. 2022(08): 160-167 . 百度学术
8.	叶来生，孟林，梁浩瀚，黄为，翁晨祎. 髋关节置换术后早期疼痛外治法应用的研究进展. 大众科技. 2022(10): 96-99 . 百度学术
9.	王琬婧，刘晓雯，刘瑶. 远红外线干预应用于自体动静脉内瘘护理的研究进展. 医疗装备. 2021(23): 194-196 . 百度学术