基于高斯模糊逻辑和ADCSCM的红外与可见光图像融合算法

李文; 叶坤涛; 舒蕾蕾; 李晟

基于高斯模糊逻辑和ADCSCM的红外与可见光图像融合算法

江西理工大学理学院，江西赣州 341000

基金项目:

江西省教育厅科学技术研究项目 GJJ170526

详细信息

作者简介:
李文（1997-），男，硕士研究生，主要研究方向为图像融合。E-mail：13986775110@163.com

通讯作者:
叶坤涛（1972-），男，博士，副教授，主要研究方向为MEMS、信号处理。E-mail：mems_123@126.com

中图分类号: TP391.41
计量
- 文章访问数: 112
- HTML全文浏览量: 42
- PDF下载量: 25
- 被引次数: 0
出版历程
- 收稿日期: 2021-05-20
- 修回日期: 2021-08-05
- 刊出日期: 2022-07-20

Infrared and Visible Image Fusion Algorithm Based on Gaussian Fuzzy Logic and Adaptive Dual-Channel Spiking Cortical Model

School of Science, Jiangxi University of Science and Technology, Ganzhou 341000, China

摘要

摘要: 为了克服当前的红外与可见光图像融合算法存在着目标不够突出、纹理细节丢失等现象，本文提出了一种基于高斯模糊逻辑和自适应双通道脉冲发放皮层模型（Adaptive Dual-Channel Spiking Cortical Model, ADCSCM）的红外与可见光图像融合算法。首先，使用非下采样剪切波变换（Non-Subsampled Sheartlet Transform, NSST）将源图像分解为低频和高频部分。其次，结合新拉普拉斯能量和（New Sum of Laplacian, NSL）与高斯模糊逻辑，设定双阈值来指导低频部分进行融合；同时，采用基于ADCSCM的融合规则来指导高频部分进行融合。最后，使用NSST逆变换进行重构来获取融合图像。实验结果表明，本文算法主观视觉效果最佳，并在互信息、信息熵和标准差3项指标上高于其他7种融合算法，能够有效突出红外目标、保留较多纹理细节，提高融合图像的质量。
- 图像融合 /
- 非下采样剪切波变换 /
- 高斯模糊逻辑 /
- 自适应双通道脉冲发放皮层模型
Abstract: To overcome the shortcomings of current infrared and visible image fusion algorithms, such as non-prominent targets and the loss of many textural details, a novel infrared and visible image fusion algorithm based on Gaussian fuzzy logic and the adaptive dual-channel spiking cortical model (ADCSCM) is proposed in this paper. First, the source infrared and visible images are decomposed into low- and high-frequency parts by non-subsampled shearlet transform (NSST). Then, these are combined with the new sum of the Laplacian and Gaussian fuzzy logic, and dual thresholds are set to guide the fusion of the low-frequency part; simultaneously, the fusion rule based on the ADCSCM is used to guide the fusion of the high-frequency part. Finally, the fused low- and high-frequency parts are reconstructed using inverse NSST to obtain the fused image. The experimental results show that the proposed algorithm has the best subjective visual effect and is better than the other seven fusion algorithms in terms of mutual information, information entropy, and standard deviation. Furthermore, the proposed algorithm can effectively highlight the infrared target, retain more textural details, and improve the quality of the fused image.
- image fusion /
- non-subsampled sheartlet transform /
- Gaussian fuzzy logic /
- adaptive dual-channel spiking cortical model

HTML全文

图 1 ADCSCM结构

Figure 1. The structure of ADCSCM

下载: 全尺寸图片幻灯片

图 2 本文算法融合流程图

Figure 2. Fusion flow chart of the proposed algorithm

下载: 全尺寸图片幻灯片

图 3 “Camp”图像的融合结果

Figure 3. Fusion results on "Camp" image

下载: 全尺寸图片幻灯片

图 4 “Lake”图像的融合结果

Figure 4. Fusion results on "Lake" image

下载: 全尺寸图片幻灯片

图 5 “Flower”图像的融合结果

Figure 5. Fusion results on "Flower" image

下载: 全尺寸图片幻灯片

图 6 “Bench”图像的融合结果

Figure 6. Fusion results on "Bench" image

下载: 全尺寸图片幻灯片

表 1 4组融合图像的客观评价结果

Table 1. The objective evaluation results of four groups of fused images

Fused images	Evaluation indexes	Algorithms
Fused images	Evaluation indexes	CVT	NSCT	GTF	NSCT-PCNN	NSST-PAPCNN	MS-WLS	MLGCF	Proposed
Camp	MI	1.3967	1.4703	1.9961	1.6344	1.9792	1.5511	1.7092	2.2472
	IE	6.5574	6.5693	6.6812	6.8681	6.8064	6.6214	6.6152	7.1566
	SF	12.2275	12.2860	8.8771	11.6638	10.5236	13.2651	12.7512	12.9934
	SD	27.1526	27.3415	27.0939	31.4014	30.2752	28.5545	29.2437	38.8731
	VIFF	0.3606	0.4256	0.2257	0.3611	0.3716	0.4692	0.4587	0.4724
Lake	MI	1.5413	1.6058	2.0167	2.1878	2.2921	2.0636	2.3721	3.9960
	IE	6.6745	6.6764	6.6217	7.2516	7.1673	7.0032	7.0096	7.4731
	SF	11.8183	11.7529	9.9321	12.2781	8.7916	12.2331	11.2376	12.1353
	SD	27.1584	27.5441	40.4490	39.8795	43.4509	35.9000	35.0912	49.5824
	VIFF	0.3260	0.3687	0.1731	0.3967	0.2784	0.4265	0.3977	0.4040
Flower	MI	3.3300	3.5893	3.1504	3.6886	3.9911	3.8955	3.9984	4.3305
	IE	6.5636	6.5577	6.2639	6.7200	6.8380	6.6259	6.6323	6.8793
	SF	20.6859	21.3924	18.3972	20.3276	19.6811	21.6768	22.4519	22.1945
	SD	36.2790	37.3107	36.3495	41.2683	41.5738	38.8609	39.5655	42.9559
	VIFF	0.7544	0.8101	0.6731	0.8449	0.8452	0.7901	0.7841	0.9164
Bench	MI	1.7790	1.8198	1.5464	3.5589	2.3970	2.3330	2.8215	3.9774
	IE	6.9686	6.9609	6.7781	7.4965	7.3619	7.1646	7.1909	7.6089
	SF	23.1776	23.2413	21.8150	23.4501	21.3190	26.3591	23.4811	23.6494
	SD	34.9933	35.2022	30.8383	59.3188	50.2010	48.4621	49.5238	63.2357
	VIFF	0.2333	0.2529	0.1260	0.2711	0.2646	0.4007	0.3662	0.2882

下载: 导出CSV

参考文献(24)

[1]	刘佳, 李登峰. 马氏距离与引导滤波加权的红外与可见光图像融合[J]. 红外技术, 2021, 43(2): 162-169. http://hwjs.nvir.cn/article/id/56484763-c7b0-4273-a087-8d672e8aba9a LIU Jia, LI Dengfeng. Infrared and visible light image fusion based on mahalanobis distance and guided filter weight-ing[J]. Infrared Technology, 2021, 43(2): 162-169. http://hwjs.nvir.cn/article/id/56484763-c7b0-4273-a087-8d672e8aba9a
[2]	江泽涛, 何玉婷, 张少钦. 一种基于对比度增强和柯西模糊函数的红外与弱可见光图像融合算法[J]. 光子学报, 2019, 48(6): 149-158. https://www.cnki.com.cn/Article/CJFDTOTAL-GZXB201906019.htm JIANG Zetao, HE Yuting, ZHANG Shaoqin. Infrared and low-light-level visible image fusion algorithm based on contrast enhancement and cauchy fuzzy function[J]. Acta Photonica Sinica, 2019, 48(6): 149-158. https://www.cnki.com.cn/Article/CJFDTOTAL-GZXB201906019.htm
[3]	MA J, ZHOU Z, WANG B, et al. Infrared and visible image fusion based on visual saliency map and weighted least square optimization[J]. Infrared Physics & Technology, 2017, 82: 8-17.
[4]	王建, 吴锡生. 基于改进的稀疏表示和PCNN的图像融合算法研究[J]. 智能系统学报, 2019, 14(5): 922-928. https://www.cnki.com.cn/Article/CJFDTOTAL-ZNXT201905011.htm WANG Jian, WU Xisheng. Image fusion based on the improved sparse representation and PCNN[J]. CAAI Transactions on Intelligent Systems, 2019, 14(5): 922-928. https://www.cnki.com.cn/Article/CJFDTOTAL-ZNXT201905011.htm
[5]	LIU Z, FENG Y, CHEN H, et al. A fusion algorithm for infrared and visible based on guided filtering and phase congruency in NSST domain[J]. Optics & Lasers in Engineering, 2017, 97: 71-77.
[6]	邓立暖, 尧新峰. 基于NSST的红外与可见光图像融合算法[J]. 电子学报, 2017, 45(12): 2965-2970. doi: 10.3969/j.issn.0372-2112.2017.12.019 DENG Linuan, YAO Xinfeng. Research on the fusion algorithm of infrared and visible images based on non-subsampled shearlet transform[J]. Acta Electronica Sinica, 2017, 45(12): 2965-2970. doi: 10.3969/j.issn.0372-2112.2017.12.019
[7]	王聪, 钱晨, 孙伟, 等. 基于SCM和CST的红外与可见光图像融合算法[J]. 红外技术, 2016, 38(5): 396-402. http://hwjs.nvir.cn/article/id/hwjs201605007 WANG Cong, QIAN Chen, SUN Wei, et al. Infrared and visible images fusion based on SCM and CST[J]. Infrared Technology, 2016, 38(5): 396-402. http://hwjs.nvir.cn/article/id/hwjs201605007
[8]	TAN W, ZHOU H, SONG J, et al. Infrared and visible image perceptive fusion through multi-level Gaussian curvature filtering image decomposition[J]. Applied Optics, 2019, 58(12): 3064-3073. doi: 10.1364/AO.58.003064
[9]	冯贺, 李立, 赵凯. 基于拉普拉斯分解耦合亮度调节的可见光与红外图像融合算法[J]. 电子测量与仪器学报, 2020, 34(10): 91-97. https://www.cnki.com.cn/Article/CJFDTOTAL-DZIY202010011.htm FENG He, LI Li, ZHAO Kai. Fusion algorithm of visible and infrared image based on Laplace decomposition coupled with brightness adjustment[J]. Journal of Electronic Measurement and Instrument, 2020, 34(10): 91-97. https://www.cnki.com.cn/Article/CJFDTOTAL-DZIY202010011.htm
[10]	GUO K, LABATE D. Optimally sparse multidimensional representation using shearlets[J]. SIAM Journal on Mathematical Analysis, 2007, 39(1): 298-318. doi: 10.1137/060649781
[11]	焦姣, 吴玲达. 向导滤波和NSST相结合的多光谱与全色图像融合算法[J]. 通信学报, 2018, 39(S2): 79-87. https://www.cnki.com.cn/Article/CJFDTOTAL-TXXB2018S2012.htm JIAO Jiao, WU Lingda. Multispectral and panchromatic images fusion method based on guided filter and NSST[J]. Journal on Communications, 2018, 39(S2): 79-87. https://www.cnki.com.cn/Article/CJFDTOTAL-TXXB2018S2012.htm
[12]	ZHAN K, ZHANG H, MA Y. New spiking cortical model for invariant texture retrieval and image processing[J]. IEEE Transactions on Neural Networks, 2009, 20(12): 1980-1986. doi: 10.1109/TNN.2009.2030585
[13]	ZHAO C, HUANG Y, QIU S. Infrared and visible image fusion algorithm based on saliency detection and adaptive double-channel spiking cortical model[J]. Infrared Physics & Technology, 2019, 102: 102976.
[14]	罗娟, 王立平, 谭云兰. 二代Curvelet变换耦合细节度量模型的遥感图像融合算法[J]. 电子测量与仪器学报, 2019, 33(7): 129-136. https://www.cnki.com.cn/Article/CJFDTOTAL-DZIY201907017.htm LUO Juan, WANG Liping, TAN Yunlan. Remote sensing image fusion method using second generation curvelet transform coupled with detail metric model[J]. Journal of Electronic Measurement and Instrument, 2019, 33(7): 129-136. https://www.cnki.com.cn/Article/CJFDTOTAL-DZIY201907017.htm
[15]	苏金凤, 张贵仓, 汪凯. 结合鲁棒主成分分析和非下采样轮廓波变换的红外与可见光图像的压缩融合[J]. 激光与光电子学进展, 2020, 57(4): 84-93. https://www.cnki.com.cn/Article/CJFDTOTAL-JGDJ202004008.htm SU J F, ZHANG G C, WANG K. Compressed fusion of infrared and visible images combining robust principal component analysis and non-subsampled contour transform[J]. Laser & Optoelectronics Progress, 2020, 57(4): 84-93. https://www.cnki.com.cn/Article/CJFDTOTAL-JGDJ202004008.htm
[16]	YIN S, CAO L, TAN Q, et al. Infrared and visible image fusion based on NSCT and fuzzy logic[C]//Proceedings of the 2010 IEEE International Conference on Mechatronics and Automation, 2010: 671-675.
[17]	朱攀, 黄战华. 基于二维经验模态分解和高斯模糊逻辑的红外与可见光图像融合[J]. 光电子·激光, 2017, 28(10): 1156-1162. https://www.cnki.com.cn/Article/CJFDTOTAL-GDZJ201710016.htm ZHU Pan, HUANG Zhanhua. Fusion of infrared and visible images based on BEMD and GFL[J]. Journal of Optoelectronics · Laser, 2017, 28(10): 1156-1162. https://www.cnki.com.cn/Article/CJFDTOTAL-GDZJ201710016.htm
[18]	闫利, 向天烛. NSCT域内结合边缘特征和自适应PCNN的红外与可见光图像融合[J]. 电子学报, 2016, 44(4): 761-766. doi: 10.3969/j.issn.0372-2112.2016.04.002 YAN Li, XIANG Tianzhu. Fusion of infrared and visible based on edge feature and adaptive PCNN in NSCT domain[J]. Acta Electronica Sinica, 2016, 44(4): 761-766. doi: 10.3969/j.issn.0372-2112.2016.04.002
[19]	侯瑞超, 周冬明, 聂仁灿, 等. 结合HSI变换与双通道脉冲发放皮层的彩色多聚焦图像融合[J]. 云南大学学报: 自然科学版, 2019, 41(2): 245-252. https://www.cnki.com.cn/Article/CJFDTOTAL-YNDZ201902006.htm HOU Ruichao, ZHOU Dongming, NIE Rencan, et al. Multi-focus color image fusion using HSI transform and dual channel spiking cortical model[J]. Journal of Yunnan University: Natural Sciences Edition, 2019, 41(2): 245-252. https://www.cnki.com.cn/Article/CJFDTOTAL-YNDZ201902006.htm
[20]	Nencini F, Garzelli A, Baronti S, et al. Remote sensing image fusion using the curvelet transform[J]. Information Fusion, 2007, 8(2): 143-156. doi: 10.1016/j.inffus.2006.02.001
[21]	ZHANG Q, Guo B. Multifocus image fusion using the nonsubsampled contourlet transform[J]. Signal Process, 2009, 89(7): 1334-1346. doi: 10.1016/j.sigpro.2009.01.012
[22]	MA J, CHEN C, LI C, et al. Infrared and visible image fusion via gradient transfer and total variation minimization[J]. Information Fusion, 2016, 31: 100-109. doi: 10.1016/j.inffus.2016.02.001
[23]	QU X, YAN J, XIAO H, et al. Image fusion algorithm based on spatial frequency-motivated pulse coupled neural networks in nonsubsampled contourlet transform domain[J]. Acta Automatica Sinica, 2008, 34(12): 1508-1514. doi: 10.1016/S1874-1029(08)60174-3
[24]	YIN M, LIU X, LIU Y, et al. Medical image fusion with parameter-adaptive pulse coupled neural network in nonsubsampled shearlet transform domain[J]. IEEE Transactions on Instrumentation and Measurement, 2019, 68(1): 49-64. doi: 10.1109/TIM.2018.2838778