基于PUCS与DTCWT的红外与弱可见光图像融合

姜迈; 沙贵君; 李宁

基于PUCS与DTCWT的红外与弱可见光图像融合

1.
中国刑事警察学院侦查与反恐怖学院，辽宁沈阳 110854
2.
中国科学院沈阳自动化研究所海洋信息技术装备中心，辽宁沈阳 110169

基金项目:

公安部技术研究计划 2020JSYJC26

公安部科技强警基础工作专项项目 2018GABJC08

中国科学院海洋信息技术创新研究院前沿基础研究项目 QYJC201913

公安理论及软科学研究计划 2019LLYJXJXY055

公安理论及软科学研究计划 2019LLYJXJXY057

详细信息

作者简介:
姜迈（1982-），男，辽宁人，博士，讲师，主要研究方向为图像处理与模式识别，三维场景重建等。E-mail: tawan_83@163.com

中图分类号: TP391.41
计量
- 文章访问数: 141
- HTML全文浏览量: 79
- PDF下载量: 33
- 被引次数: 0
出版历程
- 收稿日期: 2021-10-11
- 修回日期: 2021-11-29
- 刊出日期: 2022-07-20

Infrared and Low-level Visible Light Images Fusion Based on Perception Unified Color Space and Dual Tree Complex Wavelet Transform

1.
Criminal Investigation and Counter-terrorism College, Criminal Investigation Police University of China, Shenyang 110854, China
2.
Marine Information Technology Equipment Centre, Shenyang Institute of Automation, Shenyang 110169, China

摘要

摘要: 针对红外与弱可见光图像传统融合算法在结果图像中目标不突出、整体对比度降低、边缘及纹理细节不清晰、缺失等问题，本文提出一种基于感知一致性空间（Perception Unified Color Space，PUCS）和双树复小波变换（Dual Tree Complex Wavelet Transform，DTCWT）的融合算法。首先，将红外与弱可见光图像的亮度分量由RGB空间分别转至感知一致性空间得到新的亮度分量以备后续变换处理；接着，将源图像利用DTCWT进行多尺度分解，分别获取各自的低频分量与高频分量；然后，根据不同频带系数特点，提出一种基于区域能量自适应加权的规则对低频子带分量进行融合，采用一种基于拉普拉斯能量和与梯度值向量的规则对不同尺度、方向下高频子带分量进行融合；最后，对融合后的高、低频子带分量进行DTCWT逆变换重构图像，再将其转回至RGB空间以得到最终结果。在不同场景下将本文算法与3种高效融合算法进行对比评价，实验结果表明，本文算法不但在主观视觉上具有显著的目标特征、清晰的背景纹理及边缘细节、整体对比度适宜，而且在8项客观评价指标上也取得了较好的效果。
- 红外与弱可见光 /
- 感知一致性空间 /
- 双树复小波变换 /
- 融合规则
Abstract: To solve problems in traditional image fusion, such as dim targets, low contrast, and loss of edge and textural details in fusion results, a new fusion approach for infrared and low-level visible light image fusion based on perception unified color space (PUCS) and dual tree complex wavelet transform (DTCWT) is proposed. First, the two-source image intensity component is separately transformed from RGB space into PUCS to obtain a new intensity component for further processing. Then, the infrared and low-level visible light images are decomposed using DTCWT to obtain the low- and high-frequency components, respectively. Subsequently, at the fusion stage, the region energy adaptive weighted method is adopted to fuse the low-frequency sub-bands, and the high-frequency rule uses the sum modified Laplacian and gradient value vector for different scale and directional sub-bands fusions. Finally, the fusion image is obtained by applying inverse DTCWT on the sub-bands and returned to RGB space. The proposed algorithm was compared with three efficient fusion methods in different scenarios. The experimental results show that this approach can achieve prominent target characteristics, clear background texture and edge details, and suitable contrast in subjective evaluations as well as advantages in eight objective indicator evaluations.
- infrared and low-level visible light /
- perception unified color space /
- dual tree complex wavelet transform /
- fusion rules

HTML全文

图 1 二层分解下的DTCWT

Figure 1. Two levels of DTCWT decomposition

下载: 全尺寸图片幻灯片

图 2 四层分解下的DTCWT基函数冲击响应

Figure 2. DTCWT Basis function real and imaginary part impact response under four-levels decomposition

下载: 全尺寸图片幻灯片

图 3 基于DTCWT分解的红外与弱可见光图像

Figure 3. Infrared and low-level visible light images based on DTCWT decomposition

下载: 全尺寸图片幻灯片

图 4 基于感知一致性颜色空间与DTCWT的图像融合框架

Figure 4. Image fusion framework based on perception uniformity color space and DTCWT

下载: 全尺寸图片幻灯片

图 5 不同场景下的源红外与弱可见光图像

Figure 5. Original infrared and low-level visible light images under different scenes

下载: 全尺寸图片幻灯片

图 6 不同场景下各种算法的融合结果

Figure 6. Comparison of experimental results of different fusion algorithms under different scenes

下载: 全尺寸图片幻灯片

图 7 不同场景下融合结果客观评价指标

Figure 7. Objective evaluation indicators of fusion results under different scenes

下载: 全尺寸图片幻灯片

表 1 Q-shift滤波器组

Table 1. Q-shift filter banks

Tree A				Tree B
Real part filters		Imaginary part filters		Real part filters		Imaginary part filters
L_o1	H_i1	L_o1	H_i1	L_oq	H_iq	L_oq	H_iq
-0.0023	0.0024	0.0024	0.0023	0.0024	0.0023	-0.0023	0.0024
0.0012	-0.0013	0.0013	0.0012	0.0013	0.0012	0.0012	-0.0013
-0.0118	-0.0026	-0.0026	0.0118	-0.0026	0.0118	-0.0118	-0.0026
0.0013	0.0066	-0.0066	0.0013	-0.0066	0.0013	0.0013	0.0066
0.0444	0.0315	0.0315	-0.0444	0.0315	-0.0444	0.0444	0.0315
-0.0533	-0.0182	0.0182	-0.0533	0.0182	-0.0533	-0.0533	-0.0182
-0.1133	-0.1202	-0.1202	0.1133	-0.1202	0.1133	-0.1133	-0.1202
0.2809	-0.0246	0.0246	0.2809	0.0246	0.2809	0.2809	-0.0246
0.7528	0.5658	0.5658	-0.7528	0.5658	-0.7528	0.7528	0.5658
0.5658	-0.7528	0.7528	0.5658	0.7528	0.5658	0.5658	-0.7528
0.0246	0.2809	0.2809	-0.0246	0.2809	-0.0246	0.0246	0.2809
-0.1202	0.1133	-0.1133	-0.1202	-0.1133	-0.1202	-0.1202	0.1133
0.0182	-0.0533	-0.0533	-0.0182	-0.0533	-0.0182	0.0182	-0.0533
0.0315	-0.0444	0.0444	0.0315	0.0444	0.0315	0.0315	-0.0444
-0.0066	0.0013	0.0013	0.0066	0.0013	0.0066	-0.0066	0.00134
-0.0026	0.0118	-0.0118	-0.0026	-0.0118	-0.0026	-0.0026	0.0118
0.0013	0.0012	0.0012	-0.0013	0.0012	-0.0013	0.0013	0.0012

下载: 导出CSV

表 2 融合质量客观指标总体评价

Table 2. Overall objective evaluation indicators of different fusion methods

Evaluation indicators	Fusion algorithms
Evaluation indicators	Ours	WLS	IFE VIP	MISREG
RMSE	1	2	4	3
PFE	1	2	4	3
MAE	2	1	4	3
CORR	1	2	4	3
SNR	1	2	4	3
PSNR	1	2	4	3
MI	1	4	2	3
SSIM	1	2	4	3
Objective general evaluation	1	2	4	3
Subjective general evaluation	1	2	3	4
Execution time(s)	2.1621	6.213	0.527	8.267

下载: 导出CSV

参考文献(23)

[1]	苏金凤, 张贵仓, 汪凯. 结合RPCA和NSCT的红外与可见光图像的压缩融合[J]. 激光与光电子学进展, 2020, 57(4): 04105. https://www.cnki.com.cn/Article/CJFDTOTAL-JGDJ202004008.htm SU Jinfeng, ZHANG Guicang, WANG Kai. Infrared and visible image compressed fusion combining RPCA and NSCT[J]. Laser & Optoelectronics Progress, 2020, 57(4): 04105. https://www.cnki.com.cn/Article/CJFDTOTAL-JGDJ202004008.htm
[2]	刘明君, 董增寿. 基于多特征的红外与可见光图像融合[J]. 激光杂志, 2019, 40(10): 81-85. https://www.cnki.com.cn/Article/CJFDTOTAL-JGZZ201910017.htm LIU Mingjun, DONG Zengshou. Infrared and visible light image fusion based on multiple features[J]. Laser Journal, 2019, 40(10): 81-85. https://www.cnki.com.cn/Article/CJFDTOTAL-JGZZ201910017.htm
[3]	袁金楼, 吴谨, 刘劲. 基于NSCT与DWT的压缩感知图像融合[J]. 红外技术, 2015, 37(11): 176-182. http://hwjs.nvir.cn/article/id/hwjs201511011 YUAN Jinlou, WU Jin, LIU Jin. Image fusion based on compressed sensing of NSCT and DWT[J]. Infrared Technology, 2015, 37(11): 176-182. http://hwjs.nvir.cn/article/id/hwjs201511011
[4]	邓秋菊, 王宁. 非下采样轮廓波变换的红外与可见光图像融合[J]. 激光杂志, 2020, 41(4): 92-95. https://www.cnki.com.cn/Article/CJFDTOTAL-JGZZ202004018.htm DENG Qiuju, WANG Ning. Infrared and visible image fusion of non-down sampling wheel corridor wave transform[J]. Laser Journal, 2020, 41(4): 92-95. https://www.cnki.com.cn/Article/CJFDTOTAL-JGZZ202004018.htm
[5]	肖中杰. 基于NSCT红外与可见光融合算法优化研究[J]. 红外技术, 2017, 39(12): 1127-1130. http://hwjs.nvir.cn/article/id/hwjs201712010 XIAO Zhongjie. Improved infrared and visible light image fusion algorithm based on NSCT[J]. Infrared Technology, 2017, 39(12): 1127-1130. http://hwjs.nvir.cn/article/id/hwjs201712010
[6]	杨晟炜, 张志华, 孔玲君, 等. 基于NSST与IHS的红外与彩色可见光图像融合[J]. 包装工程, 2019, 40(11): 194-201. https://www.cnki.com.cn/Article/CJFDTOTAL-BZGC201911030.htm YANG Shengwei, ZHANG Zhihua, KONG Lingjun, et al. Fusion of infrared and color visible images based on NSST and IHS[J]. Packaging Engineering, 2019, 40(11): 194-201. https://www.cnki.com.cn/Article/CJFDTOTAL-BZGC201911030.htm
[7]	王志社, 杨风暴, 彭智浩. 基于NSST和稀疏表示的多源异类图像融合方法[J]. 红外技术, 2015, 37(3): 210-217. http://hwjs.nvir.cn/article/id/hwjs201503008 WANG Zhishe, YANG Fengbao, PENG Zhihao. Mulit-source heterogeneous image fusion based on NSST and sparse presentation[J]. Infrared Technology, 2015, 37(3): 210-217. http://hwjs.nvir.cn/article/id/hwjs201503008
[8]	叶华, 朱明旱, 王日兴. 红外和可见光图像互补融合的运动目标检测方法[J]. 红外技术, 2015, 37(8): 648-654. http://hwjs.nvir.cn/article/id/hwjs201508004 YE Hua, ZHU Minghan, WANG Rixing. Fusion of complementary information from infrared and visual images for moving object detection[J]. Infrared Technology, 2015, 37(8): 648-654. http://hwjs.nvir.cn/article/id/hwjs201508004
[9]	赵春晖, 马丽娟, 邵国锋. 采用WA-WBA与改进INSCT的图像融合算法[J]. 电子与信息学报, 2014, 36(2): 304-311. https://www.cnki.com.cn/Article/CJFDTOTAL-DZYX201402009.htm ZHAO Chunhui, MA Lijuan, SHAO Guofeng. An image fusion algorithm based on WA-WBA and improved non-subsampled contourlet transform[J]. Journal of Electronics & Information Technology, 2014, 36(2): 304-311. https://www.cnki.com.cn/Article/CJFDTOTAL-DZYX201402009.htm
[10]	杜进楷, 陈世国. 基于双树复小波变换的自适应PCNN图像融合算法[J]. 红外技术, 2018, 40(10): 1002-1007. http://hwjs.nvir.cn/article/id/hwjs201810012 DU Jinkai, CHEN Shiguo. Adaptive PCNN image fusion algorithm based on double tree complex wavelet transform[J]. Infrared Technology, 2018, 40(10): 1002-1007. http://hwjs.nvir.cn/article/id/hwjs201810012
[11]	王聪, 钱晨, 孙伟, 等. 基于SCM和CST的红外与可见光图像融合算法[J]. 红外技术, 2016, 38(5): 396-402. http://hwjs.nvir.cn/article/id/hwjs201605007 WANG Cong, QIAN Chen, SUN Wei, et al. Infrared and visible images fusion based on SCM and CST[J]. Infrared Technology, 2016, 38(5): 396-402. http://hwjs.nvir.cn/article/id/hwjs201605007
[12]	钱小燕, 张天慈, 王帮峰, 等. 局部颜色映射的彩色夜视算法[J]. 中国图象图形学报, 2012, 17(5): 689-693. https://www.cnki.com.cn/Article/CJFDTOTAL-ZGTB201205012.htm QIAN Xiaoyan, ZHANG Tianci, WANG Bangfeng, et al. Color night vision algorithm based on local color mapping[J]. Journal of Image and Graphics, 2012, 17(5): 689-693. https://www.cnki.com.cn/Article/CJFDTOTAL-ZGTB201205012.htm
[13]	Philipp Urban, Mitchell R. Rosen, ROY S. Berns. Embedding non-euclidean color spaces into euclidean color spaces with minimal isometric disagreement[J]. J. Opt. Soc. Am. A, 2007, 24(6): 1516-1528. doi: 10.1364/JOSAA.24.001516
[14]	Ingmar Lissner, Philipp Urban. Toward a unified color space for perception-based image processing[J]. IEEE Transactions on Image Processing, 2012, 21(3): 1153-1168. doi: 10.1109/TIP.2011.2163522
[15]	Nick Kingsbury. Image processing with complex wavelets[J]. Philosopical Transactions of Royal Society of London A: Mathematical, Physical and Engineering Sciences, 1999, 357(1760): 2543-2560. doi: 10.1098/rsta.1999.0447
[16]	朱攀, 刘泽阳, 黄战华. 基于DTCWT和稀疏表示的红外偏振与光强图像融合[J]. 光子学报, 2017, 46(12): 1210002-1-1210002-9. https://www.cnki.com.cn/Article/CJFDTOTAL-GZXB201712028.htm ZHU Pan, LIU Zeyang, HUANG Zhanhua, Infrared polarization and intensity image fusion based on dual-tree complex wavelet transform and space representation[J]. Acta Photonica Sinica, 2017, 46(12): 1210002-1-1210002-9. https://www.cnki.com.cn/Article/CJFDTOTAL-GZXB201712028.htm
[17]	沈瑜, 陈小朋, 杨倩. 多方向Laplacian能量和与tetrolet变换的图像融合[J]. 中国图象图形学报, 2020, 25(4): 0721-0731. https://www.cnki.com.cn/Article/CJFDTOTAL-ZGTB202004008.htm SHEN Yu, CHEN Xiaopeng, YANG Qian. Image fusion of multidirectional sum modified Laplacian and Tetrolet transform[J]. Journal of Image and Graphics, 2020, 25(4): 0721-0731. https://www.cnki.com.cn/Article/CJFDTOTAL-ZGTB202004008.htm
[18]	方静, 罗高鹏. 改进快速NSST的热烟羽红外与可见光图像融合[J]. 激光与红外, 2017, 47(7): 914-920. doi: 10.3969/j.issn.1001-5078.2017.07.024 FANG Jing, LUO Gaopeng. Infrared and visible images fusion of thermal plume based on rapid non-subsampled shearlet transform[J]. Laser & Infrared, 2017, 47(7): 914-920. doi: 10.3969/j.issn.1001-5078.2017.07.024
[19]	MA J, ZHOU Z, WANG B, et al. Infrared and visible image fusion based on visual saliency map and weighted least square optimization[J]. Infrared Physics & Technology, 2017, 82: 8-17.
[20]	ZHANG Yu, ZHANG Lijia, BAI Xiangzhi, et al. Infrared and visual Image fusion through infrared feature extraction and visual information preservation[J]. Infrared Physics & Technology, 2017, 83: 227-237.
[21]	ZHOU Zhiqiang, LI Sun, WANG Bo. Multi-scale weighted gradient-based fusion for multi-focus image[J]. Information Fusion, 2014, 20: 60-72. doi: 10.1016/j.inffus.2013.11.005
[22]	董安勇, 苏斌, 赵文博, 等. 基于卷积稀疏表示的红外与可见光图像融合[J]. 激光与红外, 2018, 48(12): 1547-1553. doi: 10.3969/j.issn.1001-5078.2018.12.018 DONG Anyong, SU Bin, ZHAO Wenbo. Infrared and visible image fusion based on convolution sparse representation[J]. Laser & Infrared, 2018, 48(12): 1547-1553. doi: 10.3969/j.issn.1001-5078.2018.12.018
[23]	周晨旭, 黄福珍. 基于BLMD和NSDFB算法的红外与可见光图像融合方法[J]. 红外技术, 2019, 41(2): 176-182. http://hwjs.nvir.cn/article/id/hwjs201902012 ZHOU Chenxu, HUANG Fuzhen. Infrared and visible image fusion based on BLMD and NSDFB[J]. Infrared Technology, 2019, 41(2): 176-182. http://hwjs.nvir.cn/article/id/hwjs201902012