Data Preprocessing Method for Infrared Spectra Analysis of Natural Gas Components
-
摘要: 利用红外光谱分析仪对天然气组分进行组分分析时所获得光谱信号往往会受杂散光、噪声、基线漂移等因素的干扰,从而影响最终定量分析结果,故需要在建模前对原始光谱进行预处理。为解决仪器测量光谱图的噪声干扰问题,本文提出一种Savitzky-Golay平滑法结合sym6小波函数软阈值去噪法对光谱图进行预处理。将传统的预处理方法与SG平滑法结合小波函数法进行对比分析。结果表明,以SG平滑法结合sym6小波函数软阈值去噪法对光谱图进行预处理,其拟合优度数值最高为0.98652,残差平方和数值最低为5.50694,证明使用该方法后的函数分峰拟合效果最佳,处理效果优于传统方法。Abstract: When using infrared spectroscopy to analyze the components of natural gas, the obtained spectral signals often contain interference from stray light, noise, baseline drift, and other factors, which affects the resulting quantitative analysis. Therefore, it is necessary to preprocess the original spectrum before modeling. As a potential solution, an SG smoothing method combined with the soft threshold denoising method of the sym6 wavelet function was proposed to preprocess the spectrogram. The traditional preprocessing method and the proposed method are compared and analyzed. The results show that when the proposed method is used to preprocess the spectrogram, the highest goodness of fit value is 0.98652, and the lowest residual sum of squares value is 5.50694, which proves that the function peak fitting effect is the best after using this method, and the processing effect is better than that of the traditional method.
-
Key words:
- natural gas /
- pretreatment /
- infrared spectra /
- denoising
-
表 1 SG平滑+sym6小波变换与传统方法性能指标对比
Table 1. Performance index comparison of SG smoothing + sym6 wavelet transform and traditional methods
Pretreatment method Fitting performance evaluation index Various fitting parameters R-Square SSE xc w FWHM A Area SG smoothing method(N=20) 0.97405 11.79833 2834.49055 39.20784 92.32725 5.59823 550.1827 2981.79713 43.97818 103.56054 2.81837 310.68118 Median filter 0.97146 14.0104 2837.10274 45.2766707 106.618 5.27621 598.804 2987.60154 37.49498 88.29445 2.94599 296.87252 SG(N=20)+db3 3-layer decomposition wavelet 0.9732 11.82987 2833.61284 38.94472 91.7078 5.60191 546.85842 2982.83007 47.12747 110.97672 2.74588 324.37366 SG(N=20)+sym6 4-layer decomposition wavelet 0.98652 5.50694 2836.78074 47.98166 112.9881 5.03243 605.2586 2988.78528 34.83569 82..03177 2.8878 252.1598 -
[1] JIAO Y, LI Z, CHEN X, et al. Preprocessing methods for near-infrared spectrum calibration[J]. Journal of Chemometrics, 2020, 34(11): e3306. [2] YANG J, DU L, GONG W, et al. Analyzing the performance of the first-derivative fluorescence spectrum for estimating leaf nitrogen concentration[J]. Optics Express, 2019, 27(4): 3978. doi: 10.1364/OE.27.003978 [3] LI Y, HUANG Y, XIA J, et al. Quantitative analysis of honey adulteration by spectrum analysis combined with several high-level data fusion strategies[J]. Vibrational Spectroscopy, 2020, 108: 103060. doi: 10.1016/j.vibspec.2020.103060 [4] 第五鹏瑶, 卞希慧, 王姿方, 等. 光谱预处理方法选择研究[J]. 光谱学与光谱分析, 2019, 39(9): 2800-2806. https://www.cnki.com.cn/Article/CJFDTOTAL-GUAN201909027.htmDIWU Pengyao, BIAN Xi Hui, WANG Zifang, et al. Selection of spectral pretreatment methods[J]. Spectroscopy and Spectral Analysis, 2019, 39 (9): 2800-2806 https://www.cnki.com.cn/Article/CJFDTOTAL-GUAN201909027.htm [5] Bastiaansen W, Klein S, Steegers㏕heunissen R P, et al. VP24.02: towards a novel human embryonic brain atlas: fully automated preprocessing of prenatal ultrasound using artificial intelligence[J]. Ultrasound in Obstetrics and Gynecology, 2020, 56(S1): 159-159. [6] HUANG X, HUANG C, ZHAI G, et al. Data processing method of multibeam bathymetry based on sparse weighted LS-SVM machine algorithm[J]. IEEE Journal of Oceanic Engineering, 2019, 45(4): 1538-1551. [7] DING Y, ZHANG W, ZHAO X, et al. A hybrid random forest method fusing wavelet transform and variable importance for the quantitative analysis of K in potassic salt or using laser-induced breakdown spectroscopy[J]. Journal of Analytical Atomic Spectrometry, 2020, 35(6): 1131-1138. doi: 10.1039/D0JA00010H [8] ZHANG J, WEN H, TANG L. Improved smoothing frequency shifting and filtering algorithm for harmonic analysis with systematic error compensation[J]. IEEE Transactions on Industrial Electronics, 2019, 66(12): 9500-9509. doi: 10.1109/TIE.2019.2892664 [9] 杨帆, 王鹏, 张宁超, 等. 一种基于小波变换的改进滤波算法及其在光谱去噪方面的应用[J]. 国外电子测量技术, 2020, 39(8): 98-104. https://www.cnki.com.cn/Article/CJFDTOTAL-GWCL202008021.htmYANG Fan, WANG Peng, ZHANG Ningchao, et al. An improved filtering algorithm based on wavelet transform and its application in spectral denoising[J]. Foreign Electronic Measurement Technology, 2020, 39 (8): 98-104. https://www.cnki.com.cn/Article/CJFDTOTAL-GWCL202008021.htm [10] Mahajan G R, Das B, Gaikwad B, et al. Monitoring properties of the salt-affected soils by multivariate analysis of the visible and near- infrared hyperspectral data[J]. Catena, 2021, 198: 105041. doi: 10.1016/j.catena.2020.105041 [11] 侯培国, 李宁, 常江, 等. SG平滑和IBPLS联合优化水中油分析方法的研究[J]. 光谱学与光谱分析, 2015, 35(6): 1529-1533. doi: 10.3964/j.issn.1000-0593(2015)06-1529-05HOU Peiguo, LI Ning, CHANG Jiang, et al. Research on SG smoothing and IBPLS joint optimization of oil-in-water analysis method[J]. Spectroscopy and Spectral Analysis, 2015, 35(6): 1529-1533. doi: 10.3964/j.issn.1000-0593(2015)06-1529-05 [12] Zikiou N, Lahdir M, Helbert D. Hyperspectral image classification using graph-based wavelet transform[J]. International Journal of Remote Sensing, 2020, 41(7): 2624-2643. doi: 10.1080/01431161.2019.1694194 [13] Bunaciu A A, Aboul-Enein H Y. Adulterated drug analysis using FTIR spectroscopy[J]. Applied Spectroscopy Reviews, 2021, 56(5): 423-437. doi: 10.1080/05704928.2020.1811717 [14] LIU F, ZHANG Y, Yildirim T, et al. Signal denoising optimization based on a Hilbert-Huang transform-triple adaptable wavelet packet transform algorithm[J]. EPL (Europhysics Letters), 2019, 124(5): 54002. doi: 10.1209/0295-5075/124/54002 [15] ZHANG M, LU C, LIU C. Improved double-threshold denoising method based on the wavelet transform[J]. OSA Continuum, 2019, 2(8): 2328-2342. doi: 10.1364/OSAC.2.002328