融合注意力分支特征的甲烷泄漏红外图像分割

何自芬; 曹辉柱; 张印辉; 黄俊璇; 史本杰; 朱守业

融合注意力分支特征的甲烷泄漏红外图像分割

昆明理工大学机电工程学院, 云南昆明 650000

基金项目:

国家青年科学基金 61302173

详细信息

作者简介:
何自芬（1976-），女，博士，副教授，硕士生导师，主要从事图像处理和机器视觉等方面的研究。E-mail：zyhhzf1998@163.com

通讯作者:
张印辉（1977-），男，博士，教授，博士生导师，主要从事图像处理、机器视觉及机器智能等方面的研究。E-mail：yinhui_z@163.com

中图分类号: TP391.4
计量
- 文章访问数: 136
- HTML全文浏览量: 131
- PDF下载量: 50
- 被引次数: 0
出版历程
- 收稿日期: 2022-04-04
- 修回日期: 2022-05-11
- 刊出日期: 2023-04-20

Infrared Image Segmentation of Methane Leaks Incorporating Attentional Branching Features

Faculty of Mechatronics and Electrical Engineering, Kunming University of Science and Technology, Kunming 650000, China

摘要

摘要: 甲烷是现代化工业生产和社会生活的重要能源之一，实现其有效探测与分割对于及时发现甲烷泄漏事故并识别其扩散范围具有重要意义。针对红外成像条件下甲烷气体图像的轮廓模糊、泄漏的甲烷气体与背景对比度较低、形状易受大气流动因素影响等问题，本文提出一种融合注意力分支特征的红外图像分割网络（Attention Branch Feature Network，ABFNet）实现甲烷气体泄漏探测。首先，为增强模型对红外甲烷气体图像的特征提取能力，设计分支特征融合模块将残差模块1和残差模块2的输出特征与残差模块3以逐像素相加的方法融合，获取红外甲烷气体图像丰富细致的特征表达以提高模型识别精度。其次，为进一步加快模型的推理速度，将标准瓶颈单元中的3×3卷积替换为深度可分离卷积，大幅度减少参数量达到实时检测甲烷气体泄漏。最后，将scSE注意力机制嵌入到分支特征融合模块，更多地关注扩散区域边缘和中心语义信息以克服红外甲烷气体轮廓模糊对比度低等问题提高模型的泛化能力。实验结果表明，本文提出的ABFNet模型AP50@95、AP50、AP60定量分割精度分别达到38.23%、89.63%和75.33%，相比于原始YOLACT模型分割精度，分别提高4.66%、3.76%和7.04%，推理速度达到34.99帧/s，满足实时检测需求。实验结果验证了本文算法对红外甲烷泄漏检测的有效性和工程实用性。
- 红外图像分割 /
- 甲烷泄漏 /
- 注意力分支特征 /
- 实时检测
Abstract: Methane is an important energy source for modern industrial production and social life, and its effective detection and segmentation are important for the timely detection of methane leaks and identification of its diffusion range. To address image problems such as blurred contours of methane gas, low contrast between leaking methane gas and the background, and susceptibility of the shape to atmospheric flow factors under infrared imaging conditions, this study proposes an infrared image segmentation network (attention branch feature network (ABFNet)) incorporating attention branch features to achieve methane gas leak detection. First, to enhance the feature extraction capability of the model for IR methane gas images, a branch feature fusion module was designed to fuse the output features of residual modules 1 and 2 with residual module 3 in a pixel-by-pixel summing method to obtain rich and detailed feature expressions of IR methane gas images to improve the model's recognition accuracy. Second, to further accelerate the inference speed of the model, the 3×3 convolution in the standard bottleneck unit was replaced with a depth-separable convolution to significantly reduce the number of parameters required for real-time methane gas leak detection. Finally, scSE attention mechanisms were embedded in the branching feature fusion module to focus more on the edge and center semantic information of the diffusion region to overcome the problem of low contrast of the blurred IR methane gas contours and improve the generalization ability of the model. The experimental results showed that the quantitative segmentation accuracy of the proposed ABFNet models AP50@95, AP50, and AP60 reached 38.23%, 89.63%, and 75.33%, respectively, with improvements of 4.66%, 3.76%, and 7.04%, respectively, compared with the segmentation accuracy of the original YOLACT model. The inference speed reached 34.99 frames/s and met the demand of real-time detection. The experimental results verified the effectiveness and engineering practicality of the proposed algorithm for infrared methane leak detection.
- infrared image segmentation /
- methane leaks /
- attentional branching features /
- real-time detection

HTML全文

图 1 ABFNet模型架构

Figure 1. ABFNet model architecture

下载: 全尺寸图片幻灯片

图 2 主干网络特征图

Figure 2. Backbone network feature map

下载: 全尺寸图片幻灯片

图 3 scSE注意力模型结构

Figure 3. scSE attention model structure

下载: 全尺寸图片幻灯片

图 4 红外甲烷数据集及真实值标签

Figure 4. Infrared methane dataset and ground truth labels

下载: 全尺寸图片幻灯片

图 5 分割结果可视化

Figure 5. Visualization of segmentation results

下载: 全尺寸图片幻灯片

表 1 FLIR GF-320红外热像仪技术参数

Table 1. Infrared thermal imaging camera technical parameters

Parameter name	Parameter value
Model number	FLIR GF320
Resolution	320×240
Thermal sensitivity	＜10 mK
Detector type	Refrigeration type indium antimonide
Wavelength range	3.2-3.4 μm

下载: 导出CSV

表 2 超参数配置

Table 2. Hyperparameter configuration

Hyperparameters	Parameter value
Initial learning rate	0.001
Weight_decay	1e-3
Learning rate decay strategy	steps_with_decay
Decay_steps	(35000, 75000, 85000)
Gamma	0.1
Number of warm-up iterations	500
Warm-up iteration strategy	Liner
Total number of iterations	110000

下载: 导出CSV

表 3 分支特征融合模块实验

Table 3. Experiment of branch feature fusion module

Models	AP50@95/%	AP50/%	AP60 /%	Inference speed/FPS
Yolact	33.57	85.87	68.29	36.18
Yolact-ABF	35.11	87.69	72.22	33.75

下载: 导出CSV

表 4 深度可分离卷积实验

Table 4. Depthwise separable convolution experiments

Models	AP50@95/%	AP50/%	AP60/%	Inference speed/FPS	Params/M
Yolact-ABF	35.11	87.69	72.22	33.75	23.50
Yolact-ABF-DWConv	35.42	87.30	69.81	37.26	13.48

下载: 导出CSV

表 5 scSE意力机制对比

Table 5. Comparison of scSE intentional force mechanisms

Embed position	AP50@95/%	AP50/%	AP60%	Inference speed/FPS
Layer1+2	35.90	87.61	71.86	35.84
Layer1+3	38.23	89.63	75.33	34.99
Layer1+4	36.67	87.80	74.76	35.19
Layer2+3	36.03	86.58	70.16	35.21
Layer2+4	35.28	84.25	73.58	34.87
Layer3+4	36.43	87.31	71.24	35.64
Layer1+2+3+4	34.13	85.61	70.12	30.15

下载: 导出CSV

表 6 消融实验

Table 6. Ablation experiments

Models	Branch feature fusion	DWConv convolution	Attention	AP50@95/%	AP50/%	AP60/%	Inference speed/FPS
YOLACT				33.57	85.87	68.29	36.18
YOLACT -ABF	√			35.11	87.69	72.22	33.75
YOLACT -DWConv		√		34.43	85.95	71.68	40.57
YOLACT -scSE			√	37.20	87.30	71.87	34.12
YOLACT -ABF- DWConv	√	√		36.64	87.72	71.24	37.26
YOLACT -ABF- scSE	√		√	35.45	86.56	67.96	34.05
YOLACT -DWConv-scSE		√	√	36.93	87.74	71.92	39.62
ABFNet	√	√	√	38.23	89.63	75.33	34.99

下载: 导出CSV

表 7 不同算法对比

Table 7. Comparison of different algorithms

Models	AP50@95/%	AP50/%	AP60/%	Inference speed/FPS
Yolact	33.57	85.87	68.29	36.18
YolactEdge	32.79	85.30	67.45	37.35
Mask R-CNN	32.60	85.20	69.37	31.23
ABFNet	38.23	89.63	75.33	34.99

下载: 导出CSV

参考文献(24)

[1]	朱亮, 高少华, 丁德武, 等. 光学气体成像技术在泄漏检测与维修中的应用研究[J]. 安全、健康和环境, 2014, 14(4): 14-16. https://www.cnki.com.cn/Article/CJFDTOTAL-SAFE201404004.htm ZHU L, GAO S H, DING D W, et al. Study on the application of optical gas imaging technology in leak detection and repair[J]. Safety, Health and Environment, 2014, 14(4): 14-16. https://www.cnki.com.cn/Article/CJFDTOTAL-SAFE201404004.htm
[2]	屈姣. 甲烷和煤尘爆炸特性实验研究[D]. 西安: 西安科技大学, 2015. QU J. Experimental study on the explosion characteristics of methane and coal dust[D]. Xi'an: Xian University of Science and Technology, 2015.
[3]	张旭, 金伟其, 李力, 等. 天然气泄漏被动式红外成像检测技术及系统性能评价研究进展[J]. 红外与激光工程, 2019, 48(S2): 53-65. https://www.cnki.com.cn/Article/CJFDTOTAL-HWYJ2019S2008.htm ZHANG X, JIN W Q, LI L, et al. Research progress on passive infrared imaging detection technology and system performance evaluation for natural gas leakage[J]. Infrared and Laser Engineering, 2019, 48(S2): 53-65. https://www.cnki.com.cn/Article/CJFDTOTAL-HWYJ2019S2008.htm
[4]	孟宗. 透射式光纤甲烷气体监测系统的研究[J]. 仪器仪表学报, 2005(S1): 49-50. https://www.cnki.com.cn/Article/CJFDTOTAL-YQXB2005S1019.htm MENG Z. Research on transmission type fiber optic methane gas monitoring system[J]. Journal of Instrumentation, 2005(S1): 49-50. https://www.cnki.com.cn/Article/CJFDTOTAL-YQXB2005S1019.htm
[5]	HU L, ZHENG C, ZHANG M, et al. Long-distance in-situ methane detection using near-infrared light-induced thermo-elastic spectroscopy[J]. Photoacoustics, 2020, 21: 100230.
[6]	苑彬, 陈书立. 红外吸收光谱型城市路网尾气浓度检测传感器[J]. 电子器件, 2020, 43(2): 471-476. doi: 10.3969/j.issn.1005-9490.2020.02.042 YUAN B, CHENG S L. Infrared absorption spectroscopy-based urban road network exhaust concentration detection sensor[J]. Electronic Devices, 2020, 43(2): 471-476. doi: 10.3969/j.issn.1005-9490.2020.02.042
[7]	迟晓铭, 肖安山, 朱亮, 等. 石化企业气体泄漏红外成像检测技术研究进展[J]. 安全、健康和环境, 2021, 21(2): 1-5. https://www.cnki.com.cn/Article/CJFDTOTAL-SAFE202102001.htm CHI X Q, XIAO AN S, ZHU L, et al. Research progress of infrared imaging detection technology for gas leakage in petrochemical enterprises[J]. Safety, Health and Environment, 2021, 21(2): 1-5. https://www.cnki.com.cn/Article/CJFDTOTAL-SAFE202102001.htm
[8]	洪少壮. 基于视觉的红外气体泄漏检测算法研究[D]. 大连: 大连海事大学, 2020. HONG S Z. Research on Infrared Gas Leak Detection Algorithm Based on Vision[D]. Dalian: Dalian Maritime University, 2020.
[9]	熊仕富. 红外热成像甲烷气体探测与识别系统关键技术研究[D]. 长春: 长春理工大学, 2018. XIONG S F. Research on Key Technologies of Infrared Thermal Imaging Methane Gas Detection and Identification System[D]. Changchun: Changchun University of Science and Technology, 2018.
[10]	王云云. 基于红外多光谱的有毒有害气体被动遥测方法研究[D]. 合肥: 中国科学技术大学, 2021. WANG Y Y. Research on Passive Telemetry of Toxic and Hazardous Gases Based on Infrared Multispectrum[D]. Hefei: University of Science and Technology of China, 2021.
[11]	方鹏, 郝宏运, 李腾飞, 等. 基于注意力机制和可变形卷积的鸡只图像实例分割提取[J]. 农业机械学报, 2021, 52(4): 257-265. https://www.cnki.com.cn/Article/CJFDTOTAL-NYJX202104027.htm FANG P, HAO H Y, LI T F, et al. Extraction of chicken image instance segmentation based on attention mechanism and deformable convolution[J]. Journal of Agricultural Machinery, 2021, 52(4): 257-265. https://www.cnki.com.cn/Article/CJFDTOTAL-NYJX202104027.htm
[12]	林云. 基于深度学习的有害气体红外图像处理研究[D]. 杭州: 浙江工商大学, 2018. LIN Y. Research on infrared image processing of harmful gases based on deep learning[D]. Hangzhou: Zhejiang University of Technology and Industry, 2018.
[13]	ZENG J, LIN Z, QI C, et al. An improved object detection method based on deep convolution neural network for smoke detection[C]// International Conference on Machine Learning and Cybernetics (ICMLC), 2018: 184-189.
[14]	Bolya D, Zhou C, Xiao F, et al. Yolact: Real-time instance segmentation[C]//Proceedings of the IEEE/CVF International Conference on Computer Vision, 2019: 9157-9166.
[15]	XIE S, Girshick R, Dollár P, et al. Aggregated residual transformations for deep neural networks[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2017: 1492-1500.
[16]	HE K M, ZHANG X Y, REN S Q, et al. Deep residual learning for image recognition[J]. IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2016: 770-778.
[17]	韩明, 王景芹, 王敬涛, 等. 级联特征融合孪生网络目标跟踪算法研究[J]. 计算机工程与应用, 2022, 58(6): 11. https://www.cnki.com.cn/Article/CJFDTOTAL-JSGG202206021.htm HAN M, WANG J J, WANG J T, et al. Research on cascaded feature fusion twin network target tracking algorithm[J]. Computer Engineering and Applications, 2022, 58(6): 11. https://www.cnki.com.cn/Article/CJFDTOTAL-JSGG202206021.htm
[18]	何自芬, 黄俊璇, 刘强, 等. 基于非对称混洗卷积神经网络的苹果叶部病害分割[J]. 农业机械学报, 2021, 52(8): 221-230. https://www.cnki.com.cn/Article/CJFDTOTAL-NYJX202108022.htm HE Z F, HUANG J X, LIU Q, et al. Apple leaf disease segmentation based on asymmetric mixed wash convolutional neural network[J]. Journal of Agricultural Machinery, 2021, 52(8): 221-230. https://www.cnki.com.cn/Article/CJFDTOTAL-NYJX202108022.htm
[19]	Chollet F. Xception: Deep learning with depthwise separable convolutions[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2017: 1251-1258.
[20]	Roy A G, Navab N, Wachinger C. Concurrent spatial and channel 'squeeze & excitation' in fully convolutional networks[C]//Medical Image Computing and Computer Assisted Intervention-MICCAI, 2018: 421-429.
[21]	HU J, SHEN L, SUN G. Squeeze-and-excitation networks[C]// Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2018: 7132-7141.
[22]	WANG J, Tchapmi L P, Ravikumar A P, et al. Machine vision for natural gas methane emissions detection using an infrared camera[J]. Applied Energy, 2020, 257: 113998.
[23]	LIU H, Soto R A R, Xiao F, et al. YOLACT edge: Real-time instance segmentation on the edge[C]//IEEE International Conference on Robotics and Automation (ICRA), 2021: 9579-9585.
[24]	HE K, Gkioxari G, Dollár P, et al. Mask R-CNN[C]//Proceedings of the IEEE International Conference on Computer Vision, 2017: 2961-2969.