Mimic Fusion Method for Differences in Dual-Mode Infrared Images
-
摘要: 针对传统融合方法无法根据双模态红外图像差异特征的不同选择有效的融合策略的问题，提出了一种面向红外光强与偏振图像差异的拟态融合方法。首先计算图像特征差异度对差异特征进行粗筛，制定主差异特征类型的选取规则来确定图像组的主差异特征；然后构造特征融合度，以建立差异特征与拟态变元集中各层变元的映射，确定变元分层结构；最后在变元分层结构选择主差异特征类型的各层变元，比较不同拟态结构变元组合时差异特征的特征融合度，确定其最大值占比最高的拟态结构，形成变体。实验结果表明，经主观分析本文方法结果的视觉效果比对比方法结果的效果更优；经客观评价本文方法结果均为有效融合，因此本文方法实现了对融合策略的自适应选择并提高了图像的融合质量。Abstract: Traditional fusion methods cannot select an effective fusion strategy based on the different characteristics of dual-mode infrared images. A mimic fusion method for the difference between the infrared intensity and polarization images was developed in this study. First, the degree of difference between image features was calculated to roughly screen the difference features, and the selection rules of the main difference feature types were formulated to determine the main difference features of the image groups. Next, the degree of feature fusion was constructed to establish the mapping between the difference features and variables in each layer of the mimic variable set and to determine the hierarchical structure of the variables. Finally, in the hierarchical structure of the variables, the variables of each layer of the main difference feature type were selected. The degrees of feature fusion of the difference features between combined variables of different mimic structures were compared to determine the mimic structure with the highest proportion of its maximum value and form a variant. The experimental results show that the visual effect of the proposed method was better than that of the comparison method after a subjective analysis. After objective evaluation, the results obtained using the proposed method indicate effective fusion. Therefore, this method realizes adaptive selection of the fusion strategy and improves image fusion quality.
-
-
![]()
图 1 面向双模态红外图像差异的拟态融合方法流程
Figure 1. Flow chart of mimic fusion method for dual-mode infrared image difference
![]()
图 2 亮度、边缘和纹理特征下的特征差异度
Figure 2. Feature difference under brightness, edge and texture features
![]()
图 5 红外光强与偏振图像融合结果
Figure 5. Fusion results of infrared light intensity and polarized images
表 1 拟态变元集
Table 1 Set of mimic variables
High-level variable Low-level variable Basic-level variable Pyramid Transform Class
Wavelet Transform Class
Directional Filtering Class
Edge Preserving ClassHigh frequency rule Low frequency rule Fusion parameter Maximum absolute value (MAX) Weighted mean (WA) Window based gradients (WBG) Window based weighted average (WBWA) Frequency selective weighted median filter (FSWM) Window based energy (WBE) Principal component analysis (PCA) Mean (MEAN) Block principal component analysis (PBPCA) Window based standard deviation (WBSD) 
下载: 导出CSV
表 2 主差异特征类型
Table 2 Main difference feature types
Image group a b c d e f Main difference Feature type AE
CD
TCDAE
TCD
DFTCD
DF
EACD
AG
TCDCD
AE
AGTCD
DF
AG
下载: 导出CSV
表 3 图像组的各层变元
Table 3 Variables of each layer of image group
Image group a b c d e f Each layer variable 1 GF
WBG_WA
symmetricGF
MAX_WBE
symmetricLP
WAX_WBE
n=4GF
MAX_WBWA
symmetricGF
MAX_WBWA
symmetricLP
MAX_WBE
n=5Each layer variable 2 GF
PCA_WBE
symmetricLP
PBPCA_WBSD
n=4NSST
MAX_WBSD
[1 2 2 4]
[32 16 16 8]DWT
MAX_WBE
n=3GF
WBG_WA
replicateNSST
MAX_WBSD
[1 2 2 4]
[32 16 16 8]Each layer variable 3 RP
MAX_WBE
n=3NSST
MAX_WBSD
[1 2 2 4]
[32 16 16 8]DTCWT
FSWM_WA
n=4LP
PBPCA_WBSD
n=4DWT
MAX_WBE
n=3DWT
MAX_WBE
n=4
下载: 导出CSV
表 4 评价指标结果
Table 4 Evaluation index results
AE STD ES EA TCR DF SF AG EN a(1) 0.4095 0.1654 0.3064 6524 0.0201 13576 0.0974 0.0316 7.3297 a(2) 0.4094 0.1650 0.2967 6310 0.0192 13083 0.0973 0.0312 7.2816 a(3) 0.4218 0.1509 0.2909 6635 0.0162 11562 0.0937 0.0305 7.1670 b(1) 0.3231 0.2182 0.2997 5136 0.0395 13957 0.0735 0.0261 7.5084 b(2) 0.3230 0.2146 0.2883 5098 0.0381 12200 0.0660 0.0249 7.4757 b(3) 0.3250 0.2135 0.2884 5369 0.0376 12180 0.0660 0.0249 7.4959 c(1) 0.3644 0.1272 0.2641 6601 0.0121 8200 0.0518 0.0224 7.0437 c(2) 0.3690 0.1248 0.2587 6386 0.0111 7813 0.0505 0.0219 7.0067 c(3) 0.3691 0.1256 0.2568 6552 0.0114 7591 0.0501 0.0217 7.0165 d(1) 0.2906 0.2262 0.4904 5818 0.0356 32283 0.1432 0.0494 7.2654 d(2) 0.2102 0.2306 0.4718 5483 0.0355 32146 0.1375 0.0480 6.9189 d(3) 0.2287 0.2398 0.4691 5741 0.0398 31247 0.1360 0.0476 7.0447 e(1) 0.5024 0.1407 0.3506 6408 0.0127 14557 0.0801 0.0327 7.0314 e(2) 0.4149 0.1600 0.3211 5237 0.0178 11267 0.0698 0.0300 7.2885 e(3) 0.4024 0.1525 0.3132 6222 0.0155 10595 0.0686 0.0295 7.1660 f(1) 0.4722 0.2289 0.1923 4261 0.0457 4636 0.0559 0.0184 7.3488 f(2) 0.4786 0.2276 0.1762 2731 0.0454 3651 0.0498 0.0167 7.2826 f(3) 0.4811 0.2273 0.1741 4423 0.0455 3395 0.0480 0.0164 7.2846
下载: 导出CSV
表 5 融合有效度结果
Table 5 Fusion effectiveness results
AE CD EA AG TCD DF a(1) 1.9966 0.0935 126019 0.8300 0.2380 314171 a(2) 1.9218 0.0598 135319 0.7991 0.1645 137245 a(3) 2.5679 0.0395 137119 0.7501 -0.1521 112802 b(1) 1.7659 0.1327 83076 0.8230 0.0526 417975 b(2) 1.6151 0.0670 95676 0.6858 -0.0927 240694 b(3) 1.6984 0.0674 106376 0.6900 -0.1431 240287 c(1) 0.5483 0.0540 117497 0.5446 0.0251 202064 c(2) 0.8172 0.0426 107697 0.4860 -0.0670 155986 c(3) 0.8526 0.0403 112597 0.4781 -0.0521 141204 d(1) 4.0119 0.2512 74796 1.3454 1.8448 479182 d(2) 0.1422 0.1253 62896 1.0272 1.8357 426562 d(3) 1.4290 0.1057 67096 1.0020 2.2600 375589 e(1) 3.8526 0.1356 69813 1.1519 0.3232 483782 e(2) -3.6600 0.0318 66713 0.6146 0.8463 122979 e(3) -4.8496 0.0252 51213 0.5991 0.5955 87527 f(1) 0.8010 0.0761 55456 0.7373 0.0571 193877 f(2) 1.3093 0.0214 48056 0.4515 0.0338 72326 f(3) 1.5717 0.0197 71656 0.4456 0.0328 69765
下载: 导出CSV
-
[1] CHEN C S, DAI M Y, LIU H F, et al. Comparison and analysis on test methods of infrared radiant intensity of infrared decoy[J]. Infrared Technology and Applications, 2014, 9300: 93000K.
[2] LI P, KANG G G. Vartiainen I. Investigation of achromatic micro polarizer array for polarization imaging in visible-infrared band[J]. Optik, 2018, 158: 1427-1435. DOI: 10.1016/j.ijleo.2018.01.017
[3] Kumar P, Gaurav A, Rajnish R K, et al. Applications of thermal imaging with infrared thermography in orthopaedics[J]. Journal of Clinical Orthopaedics and Trauma, 2021, 24: 101722.
[4] TUO H N, SHI G C, LUO X L. Detection method of ship target infrared polarization image[J]. Journal of Physics: Conference Series, 2021, 1802: 022063002E
[5] ZHANG J H, ZHANG Y, SHI Z G. Long-wave infrared polarization feature extraction and image fusion based on the orthogonality difference method[J]. Journal of Electronic Imaging, 2018, 27(2): 023021.
[6] MAJ Y, MA Y, LI C. Infrared and visible image fusion methods and applications: a survey[J]. Information Fusion, 2019, 45: 153-178. DOI: 10.1016/j.inffus.2018.02.004
[7] LI S T, KANG X D, HU J W. Image fusion with guided filtering[J]. IEEE Transactions on Image Processing, 2013, 22(7): 2864-2875. DOI: 10.1109/TIP.2013.2244222
[8] LIU Y, LIU S P, WANG Z F. A general framework for image fusion based on multi-scale transform and sparse representation[J]. Information Fusion, 2015, 24(C): 147-164.
[9] LIU Y, CHEN X, CHENG J, et al. Infrared and visible image fusion with convolutional neural networks[J]. International Journal of Wavelets Multiresolution and Information Processing, 2018, 16(3): 1850018. DOI: 10.1142/S0219691318500182
[10] HU P, YANG F B, WEI H, et al. Research on constructing difference-features to guide the fusion of dual-modal infrared images[J]. Infrared Physics & Technology, 2019, 102: 102994.
[11] 杨风暴. 红外物理与技术[M]. 2版: 北京: 电子工业出版社, 2020. YANG Fengbao. Infrared Physics and Technology[M]. 2nd edition: Beijing: Publishing House of Electronics Industry, 2020.
[12] BURT P J, ADELSON E H. The Laplacian pyramid as a compact image code[J]. IEEE Transactions on Communications, 1983, 31(4): 532-540. DOI: 10.1109/TCOM.1983.1095851
[13] Toet A, Image fusion by a ratio of low-pass pyramid[J]. Pattern Recognition Letters, 1989, 9(4): 245-253. DOI: 10.1016/0167-8655(89)90003-2
[14] Kannan1 K, Arumuga Perumal S, Arulmozhi K. Area level fusion of multi-focused images using double density DWT and DTCWT[J]. Digital Image Processing, 2009, 1(6): 231-242.
[15] 王少杰, 潘晋孝, 陈平. 基于双树复小波变换的图像融合[J]. 核电子学与探测技术, 2015, 7: 726-728. DOI: 10.3969/j.issn.0258-0934.2015.07.018 WANG S J, PAN J X, CHEN P. Image fusion based on dual-tree complex wavelet transform[J]. Nuclear Electronics and Detection Technology, 2015, 7: 726-728. DOI: 10.3969/j.issn.0258-0934.2015.07.018
[16] Easley G, Labate D, Lim W. Sparse directional image representations using the discrete shearlet transform[J]. Applied and Computational Harmonic Analysis, 2008, 25(1): 25-46. DOI: 10.1016/j.acha.2007.09.003
[17] GU B, LI W, ZHU M, et al. Local edge-preserving multiscale decomposition for high dynamic range image tone mapping[J]. IEEE Transactions on Image Processing, 2013, 22(1): 70-79. DOI: 10.1109/TIP.2012.2214047
计量
- 文章访问数: 122
- HTML全文浏览量: 23
- PDF下载量: 31
