基于结构与分解的红外光强与偏振图像融合

陈锦妮; 陈宇洋; 李云红; 拜晓桦

基于结构与分解的红外光强与偏振图像融合

西安工程大学电子信息学院, 陕西西安 710699

基金项目:

陕西省教育厅专项项目 14JK1294

陕西省科技厅一般项目 2022GY-053

陕西省科技厅一般项目 2021GY-076

详细信息

作者简介:
陈锦妮（1980-），女，博士，硕士生导师，讲师，主要从事信息与信号处理方面的研究。E-mail：chenjnxpu@163.com

通讯作者:
陈宇洋（1997-），男，硕士，学生，主要从事图像融合方面的研究。E-mail：844296749@qq.com

中图分类号: TP751.1
计量
- 文章访问数: 182
- HTML全文浏览量: 52
- PDF下载量: 46
- 被引次数: 0
出版历程
- 收稿日期: 2022-05-30
- 修回日期: 2022-07-20
- 刊出日期: 2023-03-20

Fusion of Infrared Intensity and Polarized Images Based on Structure and Decomposition

School of Electronic Information, Xi'an Polytechnic University, Xi'an 710699, China

摘要

摘要: 在一些特定环境下，红外传感器无法探测到目标时，需要将偏振技术与红外技术相融合。为了获得更清楚的融合图像，采用一种基于多尺度结构分解的图像融合方法实现红外光强与偏振图像融合。该算法提出将红外图像与偏振图分解成3个独立部分：平均强度、信号强度和信号结构。其中平均强度部分，采用一种反正切的权重函数进行融合，信号强度采用最大值的融合原则，而信号结构采用一种基于信号强度幂函数的加权平均方进行融合，最后重构得到融合图像。为了更快进行融合、降低计算的复杂度，将分解过程通过均值滤波代替，再通过上采样与下采样得到最终的融合图像。为了得到更好的融合图像，通过不同融合参数实验对比，选择较优的融合参数。最后实验表明使用所提出的反正切权重函数与融合参数设置，在与传统的多尺度算法的比较中，4项评价指标取得优势，且主观上保留更多的纹理细节、提升对比度以及抑制伪影。
- 红外偏振 /
- 多尺度结构分解 /
- 图像融合
Abstract: In specific environments, when an infrared sensor cannot detect a target, it is necessary to integrate polarization and infrared technologies. To obtain a clearer fused image, this study adopted a method based on a multiscale structure and feature image fusion to realize infrared and polarization image fusion. The algorithm decomposed the infrared image and polarization map into three independent parts: average intensity, signal intensity, and signal structure. An arctangent weight function was proposed for fusion in the average intensity part, the signal intensity adopted the maximum fusion principle, and the signal structure adopted a weighted average square based on the power function of the signal intensity for fusion, and finally, the fused image was reconstructed. To fuse faster and reduce computational complexity, the decomposition process was replaced with mean filtering, and the final fused image was obtained by upsampling and downsampling. To obtain a better fusion image, better fusion parameters were selected through an experimental comparison of different fusion parameters. Experiments showed that by using the proposed arctangent weight function and fusion parameter setting, the four evaluation indexes had advantages over the traditional multiscale algorithm and subjectively retained more texture details, improved contrast, and suppressed artifacts.
- infrared and polarization /
- multiscale structural decomposition /
- image fusion

HTML全文

图 1 控制权重函数非线性曲线

Figure 1. Nonlinear curves of control weight function

下载: 全尺寸图片幻灯片

图 2 展示多尺度结构分解算法在J＝4时的融合过程

Figure 2. Fusion process of multi-scale structural feature algorithm when J＝4

下载: 全尺寸图片幻灯片

图 3 Airport在8种多尺度融合算法的结果图。(a)红外图像；(b)偏振图像；(c)对比度金字塔；(d)离散小波变换；(e)梯度金字塔；(f)拉普拉斯金字塔；(g)主成分分析；(h)低通金字塔；(i)平移不变小波变换；(j)本文算法

Figure 3. Results of airport in 8 multiscale fusion algorithms. (a) IR; (b) DOLP; (c) CP; (d) DWT; (e) GP; (f) LP; (g) PCA; (h) RP; (i) SIDWT; (j)Proposed

下载: 全尺寸图片幻灯片

图 4 Road在8种多尺度融合算法的结果图。(a)红外图像；(b)偏振图像；(c)对比度金字塔；(d)离散小波变换；(e)梯度金字塔；(f)拉普拉斯金字塔；(g)主成分分析；(h)低通金字塔；(i)平移不变小波变换；(j)本文算法

Figure 4. Results of road in 8 multiscale fusion algorithms. (a) IR; (b) DOLP; (c) CP; (d) DWT; (e) GP; (f) LP; (g) PCA; (h) RP; (i) SIDWT; (j)Proposed

下载: 全尺寸图片幻灯片

图 5 Car在8种多尺度融合算法的结果图。(a)红外图像；(b)偏振图像；(c)对比度金字塔；(d)离散小波变换；(e)梯度金字塔；(f)拉普拉斯金字塔；(g)主成分分析；(h)低通金字塔；(i)平移不变小波变换；(j)本文算法

Figure 5. Results of car in 8 multiscale fusion algorithms. (a) IR; (b) DOLP; (c) CP; (d) DWT; (e) GP; (f) LP; (g) PCA; (h) RP; (i) SIDWT; (j)Proposed

下载: 全尺寸图片幻灯片

图 6 Windows在8种多尺度融合算法的结果图。(a)红外图像；(b)偏振图像；(c)对比度金字塔；(d)离散小波变换；(e)梯度金字塔；(f)拉普拉斯金字塔；(g)主成分分析；(h)低通金字塔；(i)平移不变小波变换；(j)本文算法

Figure 6. Results of windows in 8 multiscale fusion algorithms. (a) IR; (b) DOLP; (c) CP; (d) DWT; (e) GP; (f) LP; (g) PCA; (h) RP; (i) SIDWT; (j)Proposed

下载: 全尺寸图片幻灯片

图 7 Outdoor在8种多尺度融合算法的结果图。(a)红外图像；(b)偏振图像；(c)对比度金字塔；(d)离散小波变换；(e)梯度金字塔；(f)拉普拉斯金字塔；(g)主成分分析；(h)低通金字塔；(i)平移不变小波变换；(j)本文算法

Figure 7. Results of outdoor in 8 multiscale fusion algorithms. (a) IR; (b) DOLP; (c) CP; (d) DWT; (e) GP; (f) LP; (g) PCA; (h) RP; (i) SIDWT; (j)Proposed

下载: 全尺寸图片幻灯片

表 1 5组融合图像下不同λ的平均质量评价

Table 1. Average quality evaluation of different lambda under 5 groups of fused images

Parameter	lambda =5	lambda =10	lambda =30	lambda =60	lambda = 100	lambda = 200
EN	7.2050	7.1933	7.1731	7.1663	7.1633	7.1621
	0.5614	0.5620	0.5615	0.5609	0.5608	0.5607
SCD	1.5584	1.5400	1.5492	1.5637	1.5718	1.5785
SD	9.6199	9.5843	9.6108	9.6562	9.6403	9.6628
VIF	0.7385	0.7375	0.7364	0.7360	0.7359	0.7357
MS_SSIM	0.9661	0.9653	0.9644	0.9642	0.9643	0.9643
SSIM	0.9695	0.9691	0.9684	0.9682	0.9681	0.9680

下载: 导出CSV

表 2 融合图像的客观评价指标

Table 2. Objective evaluation indexes of fused images

Image	Evaluation	CP	DWT	GP	LP	PCA	RP	SIDWT	Proposed
Airport	EN	7.4038	6.5629	6.5538	6.5413	6.3577	7.2649	6.5136	6.4584
	Q^AB/F	0.2645	0.4154	0.4438	0.4599	0.3590	0.3114	0.4735	0.5238
	SCD	0.8556	1.1152	1.1035	1.3192	0.9929	1.3989	1.2233	0.7526
	SD	8.9371	8.4546	8.7057	8.3978	9.1176	9.4301	8.4619	8.1108
	VIF	0.7788	0.4912	0.5371	0.5760	0.5500	0.5795	0.5728	0.6035
	MS_SSIM	0.5425	0.9414	0.9518	0.9685	0.9289	0.6326	0.9752	0.9713
	SSIM	0.5021	0.9374	0.9541	0.9565	0.9217	0.5491	0.9715	0.9715
Road	EN	7.4540	7.2851	7.1242	7.4016	7.4545	7.4713	7.2958	7.7729
	Q^AB/F	0.2534	0.4296	0.4426	0.4726	0.4537	0.2949	0.4729	0.5103
	SCD	1.4472	1.6042	1.5539	1.6783	0.3136	1.5853	1.6133	1.7381
	SD	9.5398	10.4314	10.3399	10.3981	10.4773	9.9832	10.4107	10.4351
	VIF	0.9118	0.5729	0.6390	0.6850	0.9295	0.5813	0.6657	0.8105
	MS_SSIM	0.5506	0.8840	0.9042	0.9400	0.7670	0.6510	0.9382	0.9617
	SSIM	0.5906	0.9179	0.9387	0.9495	0.6648	0.6875	0.9590	0.9575
Car	EN	6.8170	6.9337	6.8247	6.9915	7.2436	6.9526	6.9374	7.4851
	Q^AB/F	0.3286	0.5521	0.5660	0.6039	0.6784	0.2854	0.6042	0.6583
	SCD	1.3841	1.4742	1.4815	1.5420	0.4592	1.4293	1.4974	1.7339
	SD	9.2526	9.6851	9.6849	9.8760	9.8916	9.6622	9.8453	9.7009
	VIF	0.5617	0.6112	0.6733	0.7692	1.0708	0.4070	0.6982	0.9140
	MS_SSIM	0.6908	0.8840	0.9042	0.9400	0.7670	0.6510	0.9382	0.9817
	SSIM	0.6762	0.9395	0.9444	0.9630	0.8894	0.7033	0.9688	0.9753
Windows	EN	7.3378	6.6612	6.5249	7.4594	7.2594	7.1577	6.6466	7.2811
	Q^AB/F	0.1769	0.4799	0.4937	0.3786	0.3686	0.2412	0.5176	0.5272
	SCD	1.1893	1.6507	1.6235	0.3339	0.3139	1.4610	1.6734	1.8535
	SD	9.1006	9.3828	9.3866	10.9746	10.9446	8.8531	9.3979	10.7113
	VIF	0.7658	0.3381	0.3302	1.0025	1.0125	0.4664	0.3704	0.4488
	MS_SSIM	0.3876	0.9059	0.9236	0.6470	0.6370	0.4996	0.9550	0.9461
	SSIM	0.4673	0.9554	0.9644	0.6249	0.6148	0.5762	0.9782	0.9688
Outdoor	EN	6.7270	6.6479	6.4716	6.7282	6.6535	6.6863	6.6336	7.0276
	Q^AB/F	0.5419	0.5050	0.5196	0.5485	0.6341	0.4022	0.5335	0.5873
	SCD	1.5299	1.5146	1.4948	1.6165	0.0819	1.6062	1.5021	1.7136
	SD	8.2680	8.5149	8.2519	8.4752	7.0202	8.1989	8.4533	9.1415
	VIF	0.7738	0.5876	0.6941	0.7670	0.9916	0.5369	0.7160	0.9157
	MS_SSIM	0.8585	0.9122	0.9260	0.9607	0.6508	0.8497	0.9586	0.9700
	SSIM	0.8927	0.9452	0.9555	0.9689	0.8242	0.8731	0.9754	0.9744
Average values of 5 fused images	EN	7.1479	6.8182	6.6998	7.0244	6.9937	7.1066	6.8054	7.2050
	Q^AB/F	0.3131	0.4764	0.4931	0.4927	0.4988	0.3070	0.5204	0.5614
	SCD	1.2812	1.4718	1.4514	1.2980	0.4323	1.4962	1.5019	1.5584
	SD	9.0196	9.2938	9.2738	9.6243	9.4903	9.2255	9.3138	9.6199
	VIF	0.7584	0.5202	0.5748	0.7599	0.9109	0.5142	0.6046	0.7385
	MS_SSIM	0.6060	0.9139	0.9252	0.8967	0.7762	0.6632	0.9573	0.9661
	SSIM	0.6258	0.9391	0.9514	0.8925	0.7830	0.6778	0.9706	0.9695

下载: 导出CSV

参考文献(23)

[1]	张肃, 付强, 段锦, 等. 基于提升小波的低对比度目标偏振识别技术[J]. 光学学报, 2015, 35(2): 0211002. https://www.cnki.com.cn/Article/CJFDTOTAL-GXXB201502017.htm ZHANG S, FU Q, DUAN J, et al. Low contrast target polarization recognition technology based on lifting wavelet[J]. Acta Optica Sinica, 2015, 35(2): 0211002. https://www.cnki.com.cn/Article/CJFDTOTAL-GXXB201502017.htm
[2]	陈潮起, 孟祥超, 邵枫, 等. 一种基于多尺度低秩分解的红外与可见光图像融合方法[J]. 光学学报, 2020, 40(11): 1110001. https://www.cnki.com.cn/Article/CJFDTOTAL-GXXB202011008.htm CHEN C, MENG X, SHAO F, et al. Infrared and visible image fusion method based on multiscale low-rank decomposition[J]. Acta Optica Sinica, 2020, 40(11): 1110001. https://www.cnki.com.cn/Article/CJFDTOTAL-GXXB202011008.htm
[3]	CHEN J, LI X J, LUO L B, et al. Infrared and visible image fusion based on target-enhanced multiscale transform decomposition[J]. Information Sciences, 2020, 508: 64-78. doi: 10.1016/j.ins.2019.08.066
[4]	汪美玉, 陈代梅, 赵根保. 基于目标提取与拉普拉斯变换的红外和可见光图像融合算法[J]. 激光与电子学进展, 2017, 54(1): 011002. https://www.cnki.com.cn/Article/CJFDTOTAL-JGDJ201701013.htm WANG M, CHEN D, ZHAO G. Image fusion algorithm of infrared and visible images based on target extraction and laplace transformation[J]. Laser & Optoelectronics Progress, 2017, 54(1): 011002. https://www.cnki.com.cn/Article/CJFDTOTAL-JGDJ201701013.htm
[5]	杨九章, 刘炜剑, 程阳. 基于对比度金字塔与双边滤波的非对称红外与可见光图像融合[J]. 红外技术, 2021, 43(9): 840-844. http://hwjs.nvir.cn/article/id/1c7de46d-f30d-48dc-8841-9e8bf3c91107 YANG J, LIU W, CHEN Y. Asymmetric infrared and visible image fusion based on contrast pyramid and bilateral filtering[J]. Infrared Technology, 2021, 43(9): 840-844. http://hwjs.nvir.cn/article/id/1c7de46d-f30d-48dc-8841-9e8bf3c91107
[6]	黄光华, 倪国强, 张彬. 一种基于视觉阈值特性的图像融合方法[J]. 北京理工大学学报, 2006(10): 907-911. doi: 10.3969/j.issn.1001-0645.2006.10.015 HUANG G, NI G, ZHANG B. Image fusion by a visual threshold based pyramid[J]. Transactions of Beijing Institute of Technology, 2006(10): 907-911. doi: 10.3969/j.issn.1001-0645.2006.10.015
[7]	李建林, 俞建成, 孙胜利. 基于梯度金字塔图像融合的研究[J]. 科学技术与工程, 2007(22): 5818-5822. https://www.cnki.com.cn/Article/CJFDTOTAL-KXJS200722017.htm LI J, YU J, SUN S. Study of image fusion based on grad pyramid algorithm[J]. Science Technology and Engineering, 2007(22): 5818-5822. https://www.cnki.com.cn/Article/CJFDTOTAL-KXJS200722017.htm
[8]	杨风暴, 董安冉, 张雷, 等. DWT、NSCT和改进PCA协同组合红外偏振图像融合[J]. 红外技术, 2017, 39(3): 201-208. http://hwjs.nvir.cn/article/id/hwjs201703001 YANG F, DONG A, ZHANG L, et al. Infrared polarization image fusion using the synergistic combination of DWT, NSCT and improved PCA[J]. Infrared Technology, 2017, 39(3): 201-208. http://hwjs.nvir.cn/article/id/hwjs201703001
[9]	安富, 杨风暴, 李伟伟, 等. 基于DWT的红外偏振与光强图像的融合[J]. 光电技术应用, 2013, 28(2): 18-22. https://www.cnki.com.cn/Article/CJFDTOTAL-GDYG201302008.htm AN F, YANG F, LI W, et al. Fusion of infrared polarization and intensity images based on DWT[J]. Electro-Optic Technology Application, 2013, 28(2): 18-22. https://www.cnki.com.cn/Article/CJFDTOTAL-GDYG201302008.htm
[10]	WANG X, WEI Y L, LIU Fu. A new multi-source image sequence fusion algorithm based on SIDWT[C]//2013 Seventh International Conference on Image and Graphics, 2013: 568-571.
[11]	YANG F, WEI H. Fusion of infrared polarization and intensity images using support value transform and fuzzy combination rules[J]. Infrared Physics & Technology, 2013, 60: 235-243.
[12]	ZHU Pan, HUANG Zhanhua. A fusion method for infrared-visible image and infrared-polarization image based on multi-scale center-surround top-hat transform[J]. Optical Review, 2017, 24(3): 1-13.
[13]	LI H, QI X B, XIE W Y. Fast infrared and visible image fusion with structural decomposition[J]. Knowledge-Based Systems, 2020, 204: 106182.
[14]	MA K, HUI L, YONG H, et al. Robust multi-exposure image fusion: a structural patch decomposition approach[J]. IEEE Transactions on Image Processing, 2017, 26(5): 2519-2532.
[15]	KOU F, LI Z G, WEN C Y, et al. Multi-scale exposure fusion via gradient domain guided image filtering[C]//2017 IEEE International Conference on Multimedia and Expo, 2017: 1105-1110.
[16]	LI H, JIA X X, ZHANG Lei. Clustering based content and color adaptive tone mapping[J]. Computer Vision and Image Understanding, 2018, 168: 37-49.
[17]	Aardt V Jan. Assessment of image fusion procedures using entropy, image quality, and multispectral classification[J]. Journal of Applied Remote Sensing, 2008, 2(1): 1-28.
[18]	Piella G, Heijmans H. A new quality metric for image fusion[C]// International Conference on Image Processing, 2003: 173-176.
[19]	Aslantas V, Bende E. A new image quality metric for image fusion: The sum of the correlations of differences[J]. AEU-International Journal of Electronics and Communications, 2015, 69(12): 1890-1896.
[20]	Altman D G, Bland J M. Statistics notes - Standard deviations and standard errors[J]. British Medical Journal, 2005, 331(7521): 903-903.
[21]	Sheikh H R, Bovik A C. Image information and visual quality[J]. IEEE Transaction on Image Processing, 2006, 15(2): 430-444.
[22]	WANG Z, Simoncelli E P, Bovik A C. Multiscale structural similarity for image quality assessment[C]//Proc IEEE Asilomar Conference on Signals, 2003: 1398-1402.
[23]	ZHOU W, Bovik A C, Sheikh H R, et al. Image quality assessment: from error visibility to structural similarity[J]. IEEE Trans Image Process, 2004, 13(4): 600-612.