基于YOLOv7-EPAN的光伏板红外图像缺陷检测

李冰; 赵宽; 白云山; 郭聪彬; 徐蔚; 徐大伟; 翟永杰

基于YOLOv7-EPAN的光伏板红外图像缺陷检测

李冰^{1, 2,},
赵宽¹,
白云山¹,
郭聪彬¹,
徐蔚¹,
徐大伟^{1, 3},
翟永杰^1, ,

1.
华北电力大学自动化系, 河北保定 071003
2.
保定市综合能源系统状态检测与优化调控重点实验室, 河北保定 071003
3.
中国科学院自动化研究所复杂系统管理与控制国家重点实验室, 北京 100190

基金项目:

国家自然科学基金项目 U21A20486

中央高校基本科研业务费专项资金资助 2022MS100

详细信息

作者简介:
李冰（1977-），男，副教授，硕士生导师，主要研究方向为模式识别与电力视觉。E-mail: li_bing_hb@126.com

通讯作者:
翟永杰（1972-），男，教授，博士生导师，主要研究方向为电力视觉。E-mail: zhaiyongjie@ncepu.edu.cn

中图分类号: TP391.41
计量
- 文章访问数: 48
- HTML全文浏览量: 4
- PDF下载量: 18
出版历程
- 收稿日期: 2023-06-30
- 修回日期: 2023-08-24
- 刊出日期: 2024-11-19

Defect Detection of Photovoltaic Panel Infrared Image Based on YOLOv7-EPAN

LI Bing^{1, 2,},
ZHAO Kuan¹,
BAI Yunshan¹,
GUO Congbin¹,
XU Wei¹,
XU Dawei^{1, 3},
ZHAI Yongjie^1, ,

1.
Department of automation, North China Electric Power University, Baoding 071003, China
2.
Baoding Key Laboratory of State Detection and Optimization Control of Integrated Energy System, Baoding 071003, China
3.
State Key Laboratory of Complex Systems Management and Control, Institute of Automation, Chinese Academy of Sciences, Beijing 100190, China

摘要

摘要:
光伏板是光伏电站重要组成部件，需定期对其进行检测，保证光伏电站安全运行。针对航拍光伏图像复杂背景下小目标难检测的问题，提出一种基于YOLOv7-EPAN的光伏板红外图像缺陷检测方法。首先提出融合CSWin Transformer的扩展高效网络CS-ELAN模块，捕获全局有效信息抑制背景信息；其次以CS-ELAN为基础构建高效路径特征聚合网络EPAN（Efficient path aggregation characteristic pyramid network），加强不同特征层的信息交互，丰富语义特征信息，提高特征表达能力；最后优化损失函数，使模型关注高质量先验框，提高小目标定位精度。在航拍光伏红外数据集上进行实验，结果表明：相比于原YOLOv7模型，所提方法的mAP₅₀、mAP_50:95分别提高了6.4%、3.3%，表明所提方法能较好地解决航拍光伏图像复杂背景下小目标缺陷漏检的问题。
- 红外图像 /
- 缺陷检测 /
- YOLOv7 /
- 深度学习 /
- CSWin Transformer /
- 小目标
Abstract:
Photovoltaic (PV) panels are an important component of photovoltaic power stations. They must be tested regularly to ensure a safe operation of the photovoltaic power station. To address the problem of small targets being difficult to detect among the complex background of aerial photovoltaic images, a defect detection method based on YOLOv7-EPAN for infrared photovoltaic panel images is proposed. First, an extended efficient network CS-ELAN module integrated with a CSWin Transformer is proposed for capturing global information effectively and suppressing background information. Second, an efficient path aggregation characteristic pyramid network (EPAN) is constructed based on CS-ELAN to enhance the information interaction between different feature layers, enrich the semantic feature information, and improve the feature expression ability. Finally, the loss function is optimized to focus the model on a prior high-quality frame and improve the positioning accuracy of small targets. The experimental results show that compared with the original YOLOv7 model, the mAP₅₀ and mAP_50:95 of the proposed method show an improvement of 6.4% and 3.3%, respectively, indicating that the proposed method can better solve the problem of missing small target defects among the complex background of aerial photovoltaic images.
- infrared images /
- defect detection /
- YOLOv7 /
- deep learning /
- CSWin Transformer /
- small target

HTML全文

图 1 YOLOv7-EPAN结构

Figure 1. YOLOv7-EPAN structure diagram

下载: 全尺寸图片幻灯片

图 2 CSWinT原理

Figure 2. CSWinT schematic diagram

下载: 全尺寸图片幻灯片

图 3 CS-ELAN结构图

Figure 3. CS-ELAN structure diagram

下载: 全尺寸图片幻灯片

图 4 光伏组件缺陷

Figure 4. Photovoltaic module defects

下载: 全尺寸图片幻灯片

图 5 标注实例框（GT）大、中、小目标的内部情况

Figure 5. Annotate the inside of large, medium, and small targets in the instance box (GT)

下载: 全尺寸图片幻灯片

图 6 不同改进策略的loss曲线

Figure 6. Loss curves for different improvement strategies

下载: 全尺寸图片幻灯片

图 7 热力图对比结果

Figure 7. Thermal image comparison results

下载: 全尺寸图片幻灯片

图 8 不同算法的检测结果

Figure 8. Detection results of different algorithms

下载: 全尺寸图片幻灯片

表 1 实验平台参数

Table 1 Experimental platform parameters

Parameters	Configuration
Operating system	Ubuntu18.04
Framework	Pytorch 1.11.0
CPU	Intel(R) Core(TM) i9-12900
GPU	NVIDIA GeForce RTX 3090 Ti
Memory	24G
Programming language	Python

下载: 导出CSV

表 2 数据集样本统计

Table 2 Data set sample statistics

Data set	Number		Category	Number of boxes
Data set	Image	Box	Category	Number of boxes
Train	2553	7668	Cell failure	5894
			Dioda failure	1367
			Occlude	407
Test	460	1566	Cell failure	1382
			Dioda failure	119
			Occlude	65

下载: 导出CSV

表 3 消融实验结果

Table 3 Ablation results %

Groups	Modules	mAP₅₀	mAP_50:95	AP^S	AP^M	params/M	FPS/(f/s)
1	Baseline	75.1	38.8	32.4	36.4	37.2	120
2	Baseline+A	80.3	40.9	35.5	36.5	35.3	83
3	Baseline+A+B	80.9	40.6	35.8	36.2	35.3	83
4	Baseline+A+C	80.6	40.3	35.6	35.2	35.3	83
5	Baseline+A+D	81.5	42.1	37.4	37.9	35.3	83
6	Baseline+A+E	79.9	40.5	36.1	35.7	35.3	83
7	Baseline+A+F	77.3	39.7	34.1	36.4	35.3	83

下载: 导出CSV

表 4 不同模型的性能对比

Table 4 Performance comparison of different models %

Modules	AP₅₀			mAP₅₀	AP^S	AP^M	Params/M	FPS/(f/s)
Modules	Cell failure	Diode failure	Occlude	mAP₅₀	AP^S	AP^M	Params/M	FPS/(f/s)
SSD	35.1	87.5	36.2	52.9	19.6	25.6	23.75	16
RetinaNet	57.6	93.4	59.3	70.1	24.3	34.1	32.24	40
YOLOv3	74.7	90.2	64.2	76.3	33.1	31.8	62.6	92
YOLOv4	77.5	92.9	47.6	72.7	30.4	35.2	63.9	97
YOLOv5s	77.2	94.0	58.9	76.7	33.6	30.8	7.02	101
YOLOv5l	77.1	91.8	65.1	78.0	34.7	37.6	46.1	62
YOLOX	79.4	93.7	63.4	78.9	-	-	8.94	89
TPH-YOLOv5	76.7	93.3	62.9	77.6	37.1	36.3	45.4	60
YOLOv7	78.7	93.9	52.6	75.1	32.4	36.4	37.2	120
YOLOv8	77.7	93.7	68.3	79.9	33.8	36.7	3.0	200
Ours	79.2	94.9	70.8	81.5	37.4	37.9	35.3	83

下载: 导出CSV

参考文献(26)

[1]	MA Q, Murshed M, Khan Z. The nexuses between energy investments, technological innovations, emission taxes, and carbon emissions in China[J]. Energy Policy, 2021, 155: 112345. DOI: 10.1016/j.enpol.2021.112345
[2]	李松浓, 晏尧, 向菲, 等. 光伏直流系统故障电弧检测方法研究综述[J/OL]. 电测与仪表: 1-9. [2023-06-04]. http://kns.cnki.net/kcms/detail/23.1202.TH.20230216.1525.010.html. LI Songnong, YAN Yao, XIANG Fei, et al. A comprehensive review on detection method for DC fault arc in photovoltaic system[J/OL]. Electrical Measurement & Instrumentation: 1-9. [2023-06-04]. http://kns.cnki.net/kcms/detail/23.1202.TH.20230216.1525.010.html.
[3]	孙建波, 王丽杰, 麻吉辉, 等. 基于改进YOLO v5s算法的光伏组件故障检测[J]. 红外技术, 2023, 45(2): 202-208. http://hwjs.nvir.cn/cn/article/id/78e76f62-17bc-444b-bac5-c2bffbef819f SUN Jianbo, WANG Lijie, MA Jihui, et al. Photovoltaic module fault detection based on improved YOLOv5s algorithm[J]. Infrared Technology, 2023, 45(2): 202-208. http://hwjs.nvir.cn/cn/article/id/78e76f62-17bc-444b-bac5-c2bffbef819f
[4]	蒋琳, 苏建徽, 施永, 等. 基于红外热图像处理的光伏阵列热斑检测方法[J]. 太阳能学报, 2020, 41(8): 180-184. DOI: 10.19912/j.0254-0096.2020.08.025. JIANG Lin, SU Jianhui, SHI Yong, et al. Hot apots detection of operating PV arrays through IR thermal image[J]. Acta Energiae Solaris Sinica, 2020, 41(8): 180-184. DOI: 10.19912/J.0254-0096.2020.08.025.
[5]	孙海蓉, 周映杰, 张镇韬, 等. 基于改进自私羊群算法的光伏红外热图像热斑识别方法[J]. 中国电机工程学报, 2022, 42(24): 8942-8951. DOI: 10.13334/j.0258-8013.pcsee.212050. SUN Hairong, ZHOU Yingjie, ZHANG Zhentao, et al. Hot spot recognition method of photovoltaic infrared thermal image based on improved selfish herd algorithm[J]. Proceedings of the CSEE, 2022, 42(24): 8942-8951. DOI:10.13334/J.0258-8013.pcse.212050.
[6]	蒋琳, 苏建徽, 李欣, 等. 基于可见光和红外热图像融合的光伏阵列热斑检测方法[J]. 太阳能学报, 2022, 43(1): 393-397. DOI: 10.19912/j.0254-0096.tynxb.2020-0142. JIANG Lin, SU Jianhui, LI Xin, et al. Hot spot detection of photovoltaic array based on fusion of visible and infrared thermal images[J]. Acta Energiae Solaris Sinica, 2022, 43(1): 393-397. DOI:10.19912/J.0254-0096.tynxb.2020-0142.
[7]	王道累, 姚勇, 张世恒, 等. 基于红外热图像的光伏组件热斑深度学习检测方法[J/OL]. 中国电机工程学报: 1-9. [2023-06-04]. https://doi.org/10.13334/j.0258-8013.pcsee.221519. WANG Daolei, YAO Yong, ZHANG Shiheng, et al. Deep learning detection method of photovoltaic module hot spot based on infrared thermal image[J/OL]. Proceedings of the CSEE: 1-9. [2023-06-04]. https://doi.org/10.13334/j.02588013.pcsee.221519.
[8]	ZHAO S, CHEN H, WANG C, et al. SNCF-Net: Scale-aware neighborhood correlation feature network for hotspot defect detection of photovoltaic farms[J]. Measurement, 2023, 206: 112342. DOI: 10.1016/j.measurement.2022.112342
[9]	刘宇宸, 李浩. 显著性特征融合的热红外图像光伏组件热斑检测[J]. 水力发电, 2023, 49(4): 96-101, 112. LIU Yuchen, LI Hao. Thermal infrared image photovoltaic hot spot detection based on saliency feature fusion[J]. Water Power, 2023, 49(4): 96-101, 112.
[10]	SU B, CHEN H, LIU K, et al. RCAG-Net: Residual channelwise attention gate network for hot spot defect detection of photovoltaic farms[J]. IEEE Transactions on Instrumentation and Measurement, 2021, 70: 1-14.
[11]	WANG C Y, Bochkovskiy A, Liao H Y M. YOLOv7: Trainable bag-of-freebies sets new state-of-the-art for real-time object detectors[J]. arXiv preprint arXiv: 2207. 02696, 2022.
[12]	宋智伟, 黄新波, 纪超, 等. 基于Flexible YOLOv7的输电线路绝缘子缺陷检测和故障预警方法[J/OL]. 高电压技术: 1-16. [2023-06-04]. https://doi.org/10.13336/j.1003-6520.hve.20221693. SONG Zhiwei, HUANG Xinbo, JI Chao, et al. Insulator defect detection and fault warning method for transmission line based on flexible YOLOV7[J/OL]. High Voltage Engineering: 1-16. [2023-06-04]. https://doi.org/10.13336/j.1003-6520.hve.20221693.
[13]	Mahrishi M, Morwal S, Muzaffar A W, et al. Video index point detection and extraction framework using custom YoloV4 Darknet object detection model[J]. IEEE Access, 2021, 9: 143378-143391. DOI: 10.1109/ACCESS.2021.3118048
[14]	DING X, ZHANG X, MA N, et al. Repvgg: Making vgg-style convnets great again[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2021: 13733-13742.
[15]	Vaswani A, Shazeer N, Parmar N, et al. Attention is all you need[J]. Advances in Neural Information Processing Systems, 2017, 30: 5998-6008.
[16]	Dosovitskiy A, Beyer L, Kolesnikov A, et al. An image is worth 16×16 words: transformers for image recognition at scale[J]. arXiv preprint arXiv: 2010. 11929, 2020.
[17]	DONG X, BAO J, CHEN D, et al. Cswin transformer: a general vision transformer backbone with cross-shaped windows[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2022: 12124-12134.
[18]	LIU Z, LIN Y, CAO Y, et al. Swin transformer: Hierarchical vision transformer using shifted windows[C]//Proceedings of the IEEE/CVF International Conference on Computer Vision, 2021: 10012-10022.
[19]	WANG J, XU C, YANG W, et al. A normalized Gaussian Wasserstein distance for tiny object detection[J]. arXiv preprint arXiv: 2110.13389, 2021.
[20]	Selvaraju R R, Cogswell M, Das A, et al. Grad-cam: visual explanations from deep networks via gradient-based localization[C]//Proceedings of the IEEE International Conference on Computer Vision, 2017: 618-626.
[21]	LIU W, Anguelov D, Erhan D, et al. SSD: single shot multibox detector[C]//Computer Vision–ECCV 2016: 14th European Conference, 2016: 21-37.
[22]	LIN T Y, Goyal P, Girshick R, et al. Focal loss for dense object detection[C]//Proceedings of the IEEE International Conference on Computer Vision, 2017: 2980-2988.
[23]	Redmon J, Farhadi A. Yolov3: an incremental improvement[J]. arXiv preprint arXiv: 1804. 02767, 2018.
[24]	Bochkovskiy A, WANG C Y, LIAO H Y M. Yolov4: Optimal speed and accuracy of object detection[J]. arXiv preprint arXiv: 2004.10934, 2020.
[25]	ZHU X, LYU S, WANG X, et al. TPH-YOLOv5: Improved YOLOv5 based on transformer prediction head for object detection on drone-captured scenarios[C]//Proceedings of the IEEE/CVF International Conference on Computer Vision, 2021: 2778-2788.
[26]	GE Z, LIU S, WANG F, et al. Yolox: Exceeding yolo series in 2021[J]. arXiv preprint arXiv: 2107.08430, 2021.