面向双模态红外图像融合算法选取的联合可能性落影构造

吴强; 吉琳娜; 杨风暴; 郭小铭

面向双模态红外图像融合算法选取的联合可能性落影构造

中北大学信息与通信工程学院, 山西太原 030000

基金项目:

山西省高等学校科技创新项目 2020L0264

国家自然科学基金 61702465

山西省应用基础研究计划 201901D211238

中北大学研究生科技立项项目 2022180501

详细信息

作者简介:
吴强（1995-），男，硕士研究生，主要研究方向为红外信息处理，E-mail: 2507066796@qq.com

通讯作者:
吉琳娜（1988-），女，副教授，硕士生导师，博士，主要研究方向为智能信息处理，Email: jlnnuc@163.com

中图分类号: TP391
计量
- 文章访问数: 116
- HTML全文浏览量: 34
- PDF下载量: 33
- 被引次数: 0
出版历程
- 收稿日期: 2022-06-02
- 修回日期: 2022-08-02
- 刊出日期: 2023-02-20

Joint Possibility Drop Shadow Construction for Selection of Bimodal Infrared Image Fusion Algorithm

College of Information and Communications Engineering, North University of China, Taiyuan 030000, China

摘要

摘要: 针对现实场景中双模态红外图像融合对异类差异特征协同优化融合的需求，且现有差异特征属性无法根据差异特征多个属性的变化针对性地调整融合算法进行有效驱动，导致融合效果差的问题，提出了面向双模态红外图像融合算法选取的联合可能性落影构造方法。首先计算双模态红外图像多融合算法下不同差异特征的融合有效度、统计差异特征分布特性；再构造差异特征融合有效度的可能性分布，通过最小二乘估计法拟合可能性分布函数；然后通过择优比较法对不同差异特征融合有效度的可能性分布进行对比分析，确定差异特征可能性分布函数投影权重，构造联合可能性落影函数；最后分析联合可能性落影函数截集水平，结合差异特征分布特性构建融合性能指标动态选取最优融合算法。实验结果表明，本文方法所选出的最优融合算法在主客观综合分析上优于其他算法，验证了本文将联合可能性落影运用于双模态红外图像最优融合算法选取中有效性和合理性。
- 双模态红外图像 /
- 融合算法选取 /
- 可能性合成 /
- 联合可能性落影
Abstract: A joint likelihood drop shadow construction method for the selection of a bimodal infrared image fusion algorithm is proposed. It aims at the demand for the cooperative and optimal fusion of dissimilar disparity features in real scenes of bimodal infrared image fusion and the limitation that the existing disparity feature attributes cannot be effectively driven by the targeted adjustment of the fusion algorithm according to the changes in multiple attributes of the disparity features, resulting in a poor fusion effect. First, we calculate the fusion effectiveness of different disparity features under the multimodal infrared image fusion algorithm and statistical disparity feature distribution characteristics. We then construct the likelihood distribution of the disparity feature fusion effectiveness and fit the likelihood distribution function by the least squares estimation method. Subsequently, we compare and analyze the likelihood distribution of different disparity feature fusion effectiveness by the merit comparison method and determine the projection weights of the disparity feature likelihood distribution function. Finally, we analyze the intercept level of the joint possibility drop shadow function and construct the optimal fusion algorithm by combining the characteristics of the distribution of different features to dynamically select the fusion performance index. The experimental results show that the optimal fusion algorithm selected in this study outperforms other algorithms in terms of subjective and objective analyses, which verifies the effectiveness and rationality of applying the joint likelihood drop shadow to the selection of an optimal fusion algorithm for bimodal infrared images.
- dual-mode infrared image /
- fusion algorithm selection /
- possible synthesis /
- joint possibility drop shadow

HTML全文

图 1 双模态红外图像融合算法选取流程

Figure 1. Flow chart of Bimodal infrared image fusion algorithm selection

下载: 全尺寸图片幻灯片

图 2 两组双模态红外源图像

Figure 2. Source dual-mode infrared images of two groups

下载: 全尺寸图片幻灯片

图 3 基于融合算法集的融合图像结果

Figure 3. Fusion image results based on fusion algorithm set

下载: 全尺寸图片幻灯片

图 4 差异特征融合有效度V_5k散点图

Figure 4. Scatter diagram of effectiveness of differential feature fusion V_5k

下载: 全尺寸图片幻灯片

图 5 差异特征可能性分布π_5k散点图

Figure 5. Differential feature possibility distribution scatter diagram

下载: 全尺寸图片幻灯片

图 6 可能性分布函数曲线Π_5k(x)(k=1, 2, 3, 4)

Figure 6. Possibility distribution function Π_5k(x)(k=1, 2, 3, 4)

下载: 全尺寸图片幻灯片

图 7 联合可能性落影函数

Figure 7. Joint possibility drop shadow function

(Note: "Dotted line": Π_5k(x)(k=1, 2, 3, 4), "Solid line": Π₅(x))

下载: 全尺寸图片幻灯片

图 8 差异特征频次分布

Figure 8. Differential characteristic frequency distribution

下载: 全尺寸图片幻灯片

图 9 总特征频次分布Ω₅^c及可能性测度权重W₅^c

Figure 9. Total feature frequency distribution Ω₅^c and probability measure weight W₅^c

下载: 全尺寸图片幻灯片

图 10 两组实验源图像以及对应融合算法的融合图像的融合效果

Figure 10. The fusion effect of two groups of experimental images and corresponding fusion algorithm

下载: 全尺寸图片幻灯片

表 1 可能性分布重要性比较权重

Table 1. Possibility distribution significance comparison weight

	Π_r1(x)	Π_r2(x)	Π_r3(x)	…	Π_rk(x)	$\sum {} $
Π_r1(x)	-	p₁₂	p₁₃	…	p_1k	$\sum\limits_{i = 1}^k {{p_{1i}}} $
Π_r2(x)	p₂₁	-	p₂₃	…	p_2k	$\sum\limits_{i = 1}^k {{p_{2i}}} $
Π_r3(x)	p₃₁	p₃₂	-	…	p_3k	$\sum\limits_{i = 1}^k {{p_{3i}}} $
$ \vdots $	$ \vdots $	$ \vdots $	$ \vdots $	-	$ \vdots $	$ \vdots $
Π_rk(x)	p_k1	p_k2	p_k3	…	-	$\sum\limits_{i = 1}^k {{p_{4i}}} $

下载: 导出CSV

表 2 可能性分布重要性比较权重

Table 2. Possibility distribution significance comparison weight

	Π_r1(x)	Π_r2(x)	Π_r3(x)	Π_r4(x)	$\sum {} $
Π_r1(x)	-	-0.3027	1.653	-0.357	0.9933
Π_r2(x)	1.3027	-	0.6645	0.3715	2.3387
Π_r3(x)	-0.653	0.3355	-	0.2277	-0.0898
Π_r4(x)	1.357	0.6285	0.7723	-	2.7578

下载: 导出CSV

表 3 2组实验图各融合算法的评价指标结果及算法排序结果

Table 3. The evaluation index results and algorithm sorting results of each fusion algorithm are shown in the experimental figure

Group	Algorithm	Evaluation index
Group	Algorithm	IE	STD	SF	AG	Q^AB/F	PSNR	MI	SSIM	指标Sr
1	DTCWT	7.1032	41.7991	17.2066	7.3088	0.4567	13.9688	4.0601	0.6254	6.5626
	DWT	6.9178	49.221	15.3076	5.9679	0.3748	13.4643	4.6401	0.5527	6.1618
	GFF	7.2938	76.9109	9.8984	3.8633	0.3348	20.0536	5.6477	0.5902	6.5204
	LAP	7.535	81.6771	9.263	3.4741	0.2606	12.9339	4.5019	0.5327	5.722
	MSVD	6.9698	46.1986	20.4573	8.5828	0.4187	13.9233	3.0158	0.608	6.6079
	WPT	7.0584	39.8688	16.8657	7.3166	0.3977	13.7724	3.8075	0.6091	6.3075
2	DTCWT	6.717	26.2934	26.2827	14.8888	0.4665	14.3888	1.4123	0.4472	6.8339
	DWT	6.3252	22.7062	9.2047	3.6809	0.1967	13.8362	3.6738	0.4315	5.2301
	GFF	6.8453	28.8261	6.0703	3.1285	0.2189	22.7237	3.6564	0.4846	5.9056
	LAP	6.3622	22.5517	4.9391	2.5674	0.1554	15.0067	2.2397	0.4544	4.613
	MSVD	6.2492	22.0069	22.9027	12.3872	0.228	14.2299	1.337	0.4096	5.7039
	WPT	6.4405	21.6285	21.3231	12.1253	0.3352	14.8892	1.294	0.3972	5.8625

下载: 导出CSV

参考文献(27)

[1]	段锦, 付强, 莫春和, 等. 国外偏振成像军事应用的研究进展(上)[J]. 红外技术, 2014, 36(3): 190-195. http://hwjs.nvir.cn/article/id/hwjs201403003 DUAN Jin, FU Qiang, MO Chunhe, et al. Review of polarization imaging technology for international military application I[J]. Infrared Technology, 2014, 36(3): 190-195. http://hwjs.nvir.cn/article/id/hwjs201403003
[2]	韩平丽. 红外辐射偏振特性及目标识别研究[D]. 西安: 西安电子科技大学, 2014. HAN Pingli. Study on Polarization Characteristics and Target Recognition of Infrared Radiation [D]. Xi 'an: Xidian University, 2014.
[3]	朱攀. 红外与红外偏振/可见光图像融合算法研究[D]. 天津: 天津大学, 2017. ZHU Pan. Study on Fusion Algorithm for Infrared and Infrared Polarization/Visible Images[D]. Tianjin: Tianjin University, 2017.
[4]	LIN Suzhen, WANG Dongjuan, ZHU Xiaohong, et al. Fusion of infrared intensity and polarization images using embedded multi-scale transform[J]. Optik-International Journal for Light and Electron Optics, 2015, 126: 5127-5133. doi: 10.1016/j.ijleo.2015.09.154
[5]	朱攀, 刘泽阳, 黄战华. 基于DTCWT和稀疏表示的红外偏振与光强图像融合[J]. 光子学报, 2017, 46(12): 213-221. https://www.cnki.com.cn/Article/CJFDTOTAL-GZXB201712028.htm ZHU Pan, LIU Zeyang, HUANG Zhanhua. Infrared polarization and light intensity image fusion based on dual-tree complex wavelet transform and sparse representation [J]. Acta Photonica Sinica, 2017, 46(12): 213-221. https://www.cnki.com.cn/Article/CJFDTOTAL-GZXB201712028.htm
[6]	LIU Z, Tsukada K, Hanasaki K, et al. Image fusion by using steerable pyramid[J]. Pattern Recognition Letters, 2001, 22(9): 929-939. doi: 10.1016/S0167-8655(01)00047-2
[7]	Vanmali A V, Gadre V M. Visible and NIR image fusion using weight-map-guided Laplacian-Gaussian pyramid for improving scene visibility[J]. Sadhana-Academy Proceedings in Engineering Sciences, 2017, 42(7): 1063-1082.
[8]	LIU Gang, JING Zhongliang, SUN Shaoyuan, et al. Image fusion based on expectation maximization algorithm and steerable pyramid[J]. Chinese Optics Letters, 2004(7): 386-389.
[9]	徐磊, 田淑昌, 崔灿, 等. 基于改进离散小波变换的多模态医学图像融合方法[J]. 中国医疗设备, 2016, 31(6): 1-6. https://www.cnki.com.cn/Article/CJFDTOTAL-CQYX201621002.htm XU Lei, TIAN Shuchang, CUI Can, et al. Multimodal medical image fusion method based on improved discrete wavelet transform[J]. China Medical Equipment, 2016, 31(6): 1-6. https://www.cnki.com.cn/Article/CJFDTOTAL-CQYX201621002.htm
[10]	HAN Xiao, ZHANG Lili, YAO Li, et al. Fusion of infrared and visible images based on discrete wavelet transform[C]//Proceedings of the Second Symposium on New Detection Technology and Its Application, National Defense Optoelectronics Forum, 2015: 38.
[11]	YAN X, QIN H, LI J, et al. Infrared and visible image fusion with spectral graph wavelet transform[J]. JOSAA, 2015, 32(9): 1643-1652. doi: 10.1364/JOSAA.32.001643
[12]	ZHAN L, ZHUANG Y, HUANG L. Infrared and visible images fusion method based on discrete wavelet transform[J]. Journal of Computers (Taiwan), 2017, 28(2): 57-71.
[13]	MA Jiayi, MA Yong, LI Chang. Infrared and visible image fusion methods and applications: a survey[J]. Information Fusion, 2019, 45: 153-178. doi: 10.1016/j.inffus.2018.02.004
[14]	杨风暴, 吉琳娜. 双模态红外图像差异特征多属性与融合算法间的深度集值映射研究[J]. 指挥控制与仿真, 2021, 43(2): 1-8. https://www.cnki.com.cn/Article/CJFDTOTAL-QBZH202102001.htm YANG Fengbao, JI Linna. Research on depth set value mapping between multi-attribute and fusion algorithm of dual-mode infrared image difference feature [J]. Command Control and Simulation, 2021, 43(2): 1-8. https://www.cnki.com.cn/Article/CJFDTOTAL-QBZH202102001.htm
[15]	张雷, 杨风暴, 吉琳娜. 差异特征指数测度的红外偏振与光强图像多算法融合[J]. 火力与指挥控制, 2018, 43(2): 49-54, 59. https://www.cnki.com.cn/Article/CJFDTOTAL-HLYZ201802011.htm ZHANG Lei, YANG Fengbao, JI Linna. Multi-algorithm fusion of infrared polarization and light intensity images based on differential feature index measure [J]. Fire Control & Command Control, 2018, 43(2): 49-54, 59. https://www.cnki.com.cn/Article/CJFDTOTAL-HLYZ201802011.htm
[16]	杨风暴, 吉琳娜, 王肖霞. 可能性理论及应用[M]. 北京: 科学出版社, 2019. YANG Fengbao, JI Linna, WANG Xiaoxia. Possibility Theory and Application [M]. Beijing: Science Press, 2019.
[17]	LIU Zhaodong, CHAI Yi, YIN Hongpeng, et al. A novel multi-focus image fusion approach based on image decomposition[J]. Information Fusion, 2017, 35: 102-116.
[18]	Toet A, Hogervorst M A. Multiscale image fusion through guided filtering[C]//Proceedings of SPIE, 2016: 99970J.
[19]	SONG Y, XIAO J, YANG J, et al. Research on MR-SVD based visual and infrared image fusion[C]//Proceedings of the International Symposium on Optoelectronic Technology and Application, 2016: 101571.
[20]	ZHANG W J, KANG J Y. QuickBird panchromatic and multi-spectral image fusion using wavelet packet transform[C]//International Conference on Intelligent Computing (ICIC), 2006, 344: 976-981.
[21]	Roberts J W, Van Aardt J, Ahmed F. Assessment of image fusion procedures using entropy image quality and multispectral classification[J]. Journal of Applied Remote Sensing, 2008, 2(1): 023522.
[22]	Eskicioglu A M, Fisher P S. Image quality measures and their performance[J]. IEEE Trans. Commun., 1995, 43(12): 2959-2965.
[23]	CUI G, FENG H, XU Z, et al. Detail preserved fusion of visible and infrared images using regional saliency extraction and multi-scale image decomposition[J]. Optics Communications, 2015, 341: 199-209.
[24]	Ratliff B M, LeMaster D A. Adaptive scene-based correction algorithm for removal of residual fixed pattern noise in microgrid image data[C]//Polarization: Measurement, Analysis, and Remote Sensing X, 2012, 8364: 83640N.
[25]	ZHOU Wang, Bovik A C, Sheikh H R, et al. Image quality assessment: from error visibility to structural similarity[J]. IEEE Transactions on Image Processing, 2004, 13(4): 600-612.
[26]	QU G, ZHANG D, YAN P. Information measure for performance of image fusion[J]. Electronics Letters, 2002, 38(7): 313-315.
[27]	ZHOU Wang, Bovik A C, Sheikh H R, et al. Image quality assessment: from error visibility to structural similarity[J]. IEEE Trans. Image Process, 2004, 13(4): 600-612.