Segmentation Method of Substation Equipment Infrared Image Based on Multimodal Image Information
-
摘要: 无人机拍摄下的红外图像中变电设备的分割精度直接影响着热故障诊断的结果，针对复杂红外背景下变电设备分割精度低的问题，提出了一种融合可见光和红外图像的多模态路径聚合网络（Multimodal Path Aggregation Network, MPAN）。首先提取并融合两种模态图像的特征，考虑到两种模态图像的特征空间存在差异，提出了自适应特征融合模块（Adaptive Feature Fuse Module, AFFM），以充分融合两种模态特征；对具有多尺度特征的主干网络增加自底向上的金字塔网络，并对横向连接的路径增强模块引入自注意力机制；最后使用dice系数优化掩膜损失函数。实验结果表明，多模态图像的融合能够增强分割性能，且验证了提出各模块的有效性，该模型能够显著提高红外图像中变电设备实例分割的准确率。Abstract: The segmentation accuracy of substation equipment in infrared images captured by a UAV directly affects the results of thermal fault diagnosis. We proposed a multimodal path aggregation network (MPAN) that fuses visible and infrared images to address the problem of low segmentation accuracy of substation equipment in complex infrared backgrounds. First, we extracted and fused the features of two modal images, and considering the differences in the feature space of the two modal images, we proposed the adaptive feature fuse module (AFFM) to fuse the two modal features fully. We added a bottom-up pyramid network to the backbone with multi-scale features and a laterally connected path enhancement. Finally, we used dice coefficients to optimize the mask loss function. The experimental results showed that the fusion of multimodal images can enhance the segmentation performance and verify the effectiveness of the proposed modules, which can significantly improve the accuracy of the segmentation of substation equipment instances in infrared images.
-
-
![]()
图 1 同一场景拍摄的可见光图像与红外图像
Figure 1. Visible light image and the infrared image is taken from the same scene
![]()
图 5 变电设备可见光图像（上）及红外图像（下）
Figure 5. Visible images (up) and infrared images (down) of substation equipment
表 1 变电设备名称及数量
Table 1 Name and quantity of substation equipment
Name Insulator Bushing Current transformer Number 1968 759 369 
下载: 导出CSV
表 2 不同算法的mAP和mAR指标对比
Table 2 Comparison of mAP and mAR metrics for different algorithms
Algorithm Backbone mAP/% mAR/% Mask R-CNN ResNet-50 58.69 59.79 Mask R-CNN ResNet-101 59.79 61.37 PANet ResNet-50 61.26 62.21 PANet ResNet-101 62.88 63.13 YOLACT ResNet-50 61.56 62.62 YOLACT ResNet-101 62.32 63.91 MPAN-RGB ResNet-50 63.37 63.96 MPAN-RGB ResNet-101 64.13 64.56 MPAN-INFRARED ResNet-50 63.69 64.13 MPAN-INFRARED ResNet-101 64.93 65.74 MPAN ResNet-50 64.78 66.87 MPAN ResNet-101 65.89 67.76
下载: 导出CSV
表 3 改进策略消融分析定量比较结果
Table 3 Quantitative comparison results of improved strategy ablation analysis
ADDM Attention Lmask_dice mAP/% mAR/% 60.17 60.86 √ 62.32 62.74 √ 61.92 62.48 √ 60.76 60.96 √ √ 64.67 65.81 √ √ 63.21 64.67 √ √ 63.82 64.43 √ √ √ 65.89 67.76
下载: 导出CSV
表 4 数据增强消融分析定量比较结果
Table 4 Quantitative comparison results of data enhancement ablation analysis
Randomly flip Color jitter Randomly rotate mAP/% mAR/% 64.68 66.37 √ 65.13 66.87 √ 64.12 64.61 √ 64.93 67.13 √ √ 65.37 66.41 √ √ 65.73 67.47 √ √ 65.48 67.04 √ √ √ 65.89 67.76
下载: 导出CSV
-
[1] 李莉, 熊炜, 陆冬梅, 等. 输变电设施可靠性评估中设备故障率预测方法研究[J]. 电测与仪表, 2015, 52(3): 37-41. https://www.cnki.com.cn/Article/CJFDTOTAL-DCYQ201503008.htm LI Li, XIONG Wei, LU Dongmei, et al. Study on the prediction method for failure rate in the reliability evaluation of power transmission and transformation facility[J]. Electrical Measurement & Instrumentation, 2015, 52(3): 37-41. https://www.cnki.com.cn/Article/CJFDTOTAL-DCYQ201503008.htm
[2] 裴少通, 刘云鹏, 陈同凡, 等. 基于BOA-SVM的劣化绝缘子红外图谱诊断方法[J]. 电测与仪表, 2018, 55(24): 11-16. https://www.cnki.com.cn/Article/CJFDTOTAL-DCYQ201824003.htm PEI Shaotong, LIU Yunpeng, CHEN Tongfan, et al. Infrared spectrum diagnosis method of deteriorated insulators based on BOA-SVM[J]. Electrical Measurement & Instrumentation, 2018, 55(24): 11-16. https://www.cnki.com.cn/Article/CJFDTOTAL-DCYQ201824003.htm
[3] 陈铁, 吕长钦, 张欣, 等. 基于KPCA-WPA-SVM的变压器故障诊断模型[J]. 电测与仪表, 2021, 58(4): 158-164. https://www.cnki.com.cn/Article/CJFDTOTAL-DCYQ202104023.htm CHEN Tie, LYU Changqin, ZHANG Xin, et al. Transformer fault diagnosis model based on KPCA-WPA-SVM[J]. Electrical Measurement & Instrumentation, 2021, 58(4): 158-164. https://www.cnki.com.cn/Article/CJFDTOTAL-DCYQ202104023.htm
[4] Furse C M, Kafal M, Razzaghi R, et al. Fault diagnosis for electrical systems and power networks: A review[J]. IEEE Sensors Journal, 2020, 21(2): 888-906.
[5] Ferreira V H, Zanghi R, Fortes M Z, et al. A survey on intelligent system application to fault diagnosis in electric power system transmission lines[J]. Electric Power Systems Research, 2016, 136: 135-153. DOI: 10.1016/j.epsr.2016.02.002
[6] 王小芳, 毛华敏. 一种复杂背景下的电力设备红外图像分割方法[J]. 红外技术, 2019, 41(12): 1111-1116. http://hwjs.nvir.cn/article/id/hwjs201912004 WANG Xiaofang, MAO Huamin. Infrared image segmentation method for power equipment in complex background[J]. Infrared Technology, 2019, 41(12): 1111-1116. http://hwjs.nvir.cn/article/id/hwjs201912004
[7] 王晓飞, 胡凡奎, 黄硕. 基于分布信息直觉模糊c均值聚类的红外图像分割算法[J]. 通信学报, 2020, 41(5): 120-129. https://www.cnki.com.cn/Article/CJFDTOTAL-TXXB202005013.htm WANG Xiaofei, HU Fankui, HUANG Shuo. Infrared image segmentation algorithm based on distribution information intuitionistic fuzzy c-means clustering[J]. Journal on Communications, 2020, 41(5): 120-129. https://www.cnki.com.cn/Article/CJFDTOTAL-TXXB202005013.htm
[8] 冯振新, 许晓路, 周东国, 等. 基于局部区域聚类的电力设备故障区域提取方法[J]. 电测与仪表, 2020, 57(8): 45-50. https://www.cnki.com.cn/Article/CJFDTOTAL-DCYQ202008008.htm FENG Zhenxin, XU Xiaolu, Zhou Dongguo, et al. Extraction method of power device fault region based on local clustering algorithm[J]. Electrical Measurement & Instrumentation, 2020, 57(8): 45-50. https://www.cnki.com.cn/Article/CJFDTOTAL-DCYQ202008008.htm
[9] 刘云鹏, 张喆, 裴少通, 等. 基于深度学习的红外图像中劣化绝缘子片的分割方法[J]. 电测与仪表, 2022, 59(9): 63-68, 118. Doi: 10.19753/j.issn1001-1390.2022.09.009. LIU Yunpeng, ZHANG Zhe, PEI Shaotong, et al. Faulty insulator segmentation method in infrared image based on deep learning[J]. Electrical Measurement & Instrumentation, 2022, 59(9): 63-68, 118. Doi: 10.19753/j.issn1001-1390.2022.09.009.
[10] 吴克河, 王敏鉴, 李渊博. 基于Mask R-CNN的电力设备红外图像分割技术研究[J]. 计算机与数字工程, 2020, 48(2): 417-422. https://www.cnki.com.cn/Article/CJFDTOTAL-JSSG202306005.htm WU Kehe, WANG Minjian, LI Yuanbo. Research on infrared image segmentation technology of power equipment based on Mask R-CNN[J]. Computer & Digital Engineering, 2020, 48(2): 417-422. https://www.cnki.com.cn/Article/CJFDTOTAL-JSSG202306005.htm
[11] 李文璞, 毛颖科, 廖逍, 等. 基于旋转目标检测的变电设备红外图像电压致热型缺陷智能诊断方法[J]. 高电压技术, 2021, 47(9): 3246-3253. https://www.cnki.com.cn/Article/CJFDTOTAL-GDYJ202109022.htm LI Wenpu, MAO Yingke, LIAO Xiao, et al. Intelligent diagnosis method of infrared image for substation equipment voltage type thermal defects based on rotating target detection[J]. High Voltage Engineering, 2021, 47(9): 3246-3253. https://www.cnki.com.cn/Article/CJFDTOTAL-GDYJ202109022.htm
[12] BOLYA D, ZHOU C, XIAO F, et al. YOLACT: real-time instance segmentation[J/OL]. Computer Vision and Pattern Recognition, 2019, https://arxiv.org/abs/1904.02689 2019: 1904-2689.
[13] 田乐, 王欢. 引入独立融合分支的双模态语义分割网络[J]. 计算机工程, 2022, 48(8): 240-248, 257. Doi: 10.19678/j.issn.1000-3428.0062066. TIAN Le, WANG Huan. Dual-modal semantical segmentation network by involving independent fusion branch[J]. Computer Engineering, 2022, 48(8): 240-248, 257. Doi: 10.19678/j.issn.1000-3428.0062066.
[14] CHEN L C, Papandreou G, Kokkinos I, et al. DeepLab: Semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected CRFs[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2018, 40(4): 834-848
[15] LIN T, DOLLÁR P, GIRSHICK R, et al. Feature pyramid networks for object detection[C]//IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2017: 936-944, Doi: 10.1109/CVPR.2017.106.
[16] 胡志伟, 杨华, 娄甜田. 采用双重注意力特征金字塔网络检测群养生猪[J]. 农业工程学报, 2021, 37(5): 166-174. https://www.cnki.com.cn/Article/CJFDTOTAL-NYGU202105019.htm HU Zhiwei, YANG Hua, LOU Tiantian. Instance detection of group breeding pigs using a pyramid network with dual attention feature[J]. Transactions of the Chinese Society of Agricultural Engineering, 2021, 37(5): 166-174. https://www.cnki.com.cn/Article/CJFDTOTAL-NYGU202105019.htm
[17] CHENG B, Girshick R, Dollár P, et al. Boundary IoU: improving object-centric image segmentation evaluation[J/OL]. Computer Vision and Pattern Recognition, 2021: 2103-16562. https://arxiv.org/abs/2103.16562.
计量
- 文章访问数: 130
- HTML全文浏览量: 52
- PDF下载量: 35
