Defect Detection of Photovoltaic Panel Based on Multisource Image Fusion

YAN Hao; DAI Jiajia; GONG Xiaoxi; WU Yuxiang; WANG Jun

Volume 45 Issue 5

May 2023

Turn off MathJax

Article Contents

Article Navigation > Infrared Technology > 2023 > 45(5): 488-497

YAN Hao, DAI Jiajia, GONG Xiaoxi, WU Yuxiang, WANG Jun. Defect Detection of Photovoltaic Panel Based on Multisource Image Fusion[J]. Infrared Technology , 2023, 45(5): 488-497.

Citation:

YAN Hao, DAI Jiajia, GONG Xiaoxi, WU Yuxiang, WANG Jun. Defect Detection of Photovoltaic Panel Based on Multisource Image Fusion[J]. Infrared Technology , 2023, 45(5): 488-497.

YAN Hao, DAI Jiajia, GONG Xiaoxi, WU Yuxiang, WANG Jun. Defect Detection of Photovoltaic Panel Based on Multisource Image Fusion[J]. Infrared Technology , 2023, 45(5): 488-497.

Citation:

YAN Hao, DAI Jiajia, GONG Xiaoxi, WU Yuxiang, WANG Jun. Defect Detection of Photovoltaic Panel Based on Multisource Image Fusion[J]. Infrared Technology , 2023, 45(5): 488-497.

PDF( 6443 KB)

Defect Detection of Photovoltaic Panel Based on Multisource Image Fusion

College of Mechanical & Electrical Engineering, Nanjing University of Aeronautics and Astronautics, Nanjing 210006, China

Received Date: 2022-08-17
Rev Recd Date: 2022-09-13
Publish Date: 2023-05-20

Abstract

Abstract

This study proposed a multisource fusion network (MF-Net) that combines visible and infrared images for the inspection of a photovoltaic panel to achieve photovoltaic panel defect detection, defect classification, and localization. The limitations of the traditional methods include low efficiency, low accuracy, and high cost. In this study, a defect detection network was designed based on the backbone of YOLOv3-tiny. Deep layers are added to the network, constituting a dense block structure to augment semantic information on fused feature maps. The detection scale of the network was extended to improve its applicability at different scales. In addition, an adaptive weight fusion strategy was proposed to achieve feature map fusion, where the fusion coefficients can be allocated according to the pixel neighborhood information. Compared with the backbone, the results show that the mAP of our network improved by 7.41%. The performance improves (by approximately 2.14% mAP) when the weighted fusion strategy is replaced with ours, and the significance of the features can be effectively improved. Relative to other networks, the proposed network that takes the fused images as the input has the highest performance in terms of the F1 score (F1=0.86).
- photovoltaic panel,
- image fusion,
- defect detection,
- visible image,
- infrared image

FullText(HTML)

References(23)

References

[1]	Alves R H F, Deus Júnior G A, Marra E G, et al. Automatic fault classification in photovoltaic modules using Convolutional Neural Networks[J]. Renewable Energy, 2021, 179: 502-516. doi: 10.1016/j.renene.2021.07.070
[2]	闫冯渊. 实用化光伏组件监测电路研制[D]. 北京: 华北电力大学, 2021. YAN Fengyuan. Development of Practical Monitoring Circuit For Photovoltaic Modules[D]. Beijing: North China Electric Power University, 2021.
[3]	林剑春, 杨爱军, 沈熠辉. 电致发光缺陷检测仪的成像性能评估[J]. 光学精密工程, 2017, 25(6): 1418-1424. https://www.cnki.com.cn/Article/CJFDTOTAL-GXJM201706003.htm LIN Jianchun, YANG Aijun, SHEN Yihui. Evaluation of imaging performance for electroluminescence defect detector[J]. Optics and Precision Engineering, 2017, 25(6): 1418-1424. https://www.cnki.com.cn/Article/CJFDTOTAL-GXJM201706003.htm
[4]	LI X, LI W, YANG Q, et al. An unmanned inspection system for multiple defects detection in photovoltaic plants[J]. IEEE Journal of Photovoltaics, 2020, 10(2): 568-576. doi: 10.1109/JPHOTOV.2019.2955183
[5]	Espinosa A R, Bressan M, Giraldo L F. Failure signature classification in solar photovoltaic plants using RGB images and convolutional neural networks – Science Direct[J]. Renewable Energy, 2020, 162: 249-256. doi: 10.1016/j.renene.2020.07.154
[6]	邓堡元, 何赟泽, 王洪金, 等. 光伏电池图像序列的深度学习检测方法[J]. 机械工程学报, 2021, 57(8): 98-106. https://www.cnki.com.cn/Article/CJFDTOTAL-JXXB202108009.htm DENG Baoyuan, HE Yunze, WANG Hongjin, et al. Deep learning inspection for photovoltaic cell image sequence[J]. Journal of Mechanical Engineering, 2021, 57(8): 98-106. https://www.cnki.com.cn/Article/CJFDTOTAL-JXXB202108009.htm
[7]	孙智权, 周奇, 陈震, 等. 基于CMOS图像传感器的太阳能电池缺陷检测系统设计[J]. 仪表技术与传感器, 2018(1): 60-63. doi: 10.3969/j.issn.1002-1841.2018.01.015 SUN Zhiquan, ZHOU Qi, CHEN Zhen, et al. Design of solar cell defects detection system based on CMOS image sensor[J]. Instrument Technique and Sensor, 2018(1): 60-63. doi: 10.3969/j.issn.1002-1841.2018.01.015
[8]	HE Y, DU B, HUANG S. Noncontact electromagnetic induction excited infrared thermography for photovoltaic cells and modules inspection[J]. IEEE Transactions on Industrial Informatics, 2018, 14(12): 5585-5593. doi: 10.1109/TII.2018.2822272
[9]	郭梦浩, 徐红伟. 基于Faster RCNN的红外热图像热斑缺陷检测研究[J]. 计算机系统应用, 2019, 28(11): 265-270. https://www.cnki.com.cn/Article/CJFDTOTAL-XTYY201911039.htm GUO Menghao, XU Hongwei. Hot spot defect detection based on infrared thermal image and Faster RCNN[J]. Computer Systems & Applications, 2019, 28(11): 265-270. https://www.cnki.com.cn/Article/CJFDTOTAL-XTYY201911039.htm
[10]	谢春宇. 基于红外与可见光双模图像融合的目标检测跟踪技术研究[D]. 南京: 东南大学, 2020. XIE Chunyu. Research on target detection and tracking technology based on infrared and visible dual mode image fusion[D]. Nanjing: Southeast University, 2020.
[11]	SONG H, LIU Z, DU H, et al. Depth-aware salient object detection and segmentation via multiscale discriminative saliency fusion and bootstrap learning[J]. IEEE Transactions on Image Processing, 2017, 26(9): 4204-4216. doi: 10.1109/TIP.2017.2711277
[12]	郑欣悦, 赖际舟, 吕品, 等. 基于红外视觉/激光雷达融合的目标识别与定位方法[J]. 导航定位与授时, 2021, 8(3): 34-41. https://www.cnki.com.cn/Article/CJFDTOTAL-DWSS202103007.htm ZHENG Xinyue, LAI Jizhou, LV Pin, et al. Object detection and positioning method based on infrared vision/lidar fusion[J]. Navigation Positioning & Timing, 2021, 8(3): 34-41. https://www.cnki.com.cn/Article/CJFDTOTAL-DWSS202103007.htm
[13]	康硕, 柯臻铮, 王璇, 等. 基于红外和可见光图像融合的铺丝缺陷检测方法[J]. 航空学报, 2021, 42(12): 425187. https://www.cnki.com.cn/Article/CJFDTOTAL-HKXB202203045.htm KANG Shuo, KE Zhenzheng, WANG Xuan, et al. Detection method of defects in automatic fiber placement based on infrared and visible image fusion[J]. Acta Aeronautica et Astronautica Sinica, 2021, 42(12): 425187. https://www.cnki.com.cn/Article/CJFDTOTAL-HKXB202203045.htm
[14]	LIU D, ZHOU D, NIE R, et al. Infrared and visible image fusion based on convolutional neural network model and saliency detection via hybrid l0-l1 layer decomposition[J]. Journal of Electronic Imaging, 2018, 27(6): 063036. http://www.researchgate.net/publication/329955538_Infrared_and_visible_image_fusion_based_on_convolutional_neural_network_model_and_saliency_detection_via_hybrid_l0-l1_layer_decomposition
[15]	马旗, 朱斌, 程正东, 等. 基于双通道的快速低空无人机检测识别方法[J]. 光学学报, 2019, 39(12): 105-115. https://www.cnki.com.cn/Article/CJFDTOTAL-GXXB201912012.htm MA Qi, ZHU Bin, CHENG Zhengdong, et al. Detection and recognition method of fast low-altitude unmanned aerial vehicle based on dual channel[J]. Acta Optica Sinica, 2019, 39(12): 105-115. https://www.cnki.com.cn/Article/CJFDTOTAL-GXXB201912012.htm
[16]	白玉, 侯志强, 刘晓义, 等. 基于可见光图像和红外图像决策级融合的目标检测算法[J]. 空军工程大学学报: 自然科学版, 2020, 21(6): 53-59, 100. https://www.cnki.com.cn/Article/CJFDTOTAL-KJGC202006009.htm BAI Yu, HOU Zhiqiang, LIU Xiaoyi, et al. An object detection algorithm based on decision-level fusion of visible light image and infrared image[J]. Journal of Air Force Engineering University: Natural Science Edition, 2020, 21(6): 53-59, 100. https://www.cnki.com.cn/Article/CJFDTOTAL-KJGC202006009.htm
[17]	唐聪, 凌永顺, 杨华, 等. 基于深度学习的红外与可见光决策级融合检测(英文)[J]. 红外与激光工程, 2019, 48(6): 456-470. https://www.cnki.com.cn/Article/CJFDTOTAL-HWYJ201906045.htm TANG Cong, LING Yongshun, YANG Hua, et al. Decision-level fusion detection for infrared and visible spectra based on deep learning[J]. Infrared and Laser Engineering, 2019, 48(6): 456-470. https://www.cnki.com.cn/Article/CJFDTOTAL-HWYJ201906045.htm
[18]	刘家银, 印杰, 牛博威, 等. 海量网站中博彩类违法网站的捕获方法[J]. 数据采集与处理, 2021, 36(5): 1050-1061. https://www.cnki.com.cn/Article/CJFDTOTAL-SJCJ202105021.htm LIU Jiayin, YIN Jie, NIU Bowei, et al. Capture methods of gambling related illegal websites in massive websites[J]. Journal of Data Acquisition and Processing, 2021, 36(5): 1050-1061. https://www.cnki.com.cn/Article/CJFDTOTAL-SJCJ202105021.htm
[19]	DING S, LONG F, FAN H, et al. A novel YOLOv3-tiny network for unmanned airship obstacle detection[C]// IEEE 8th Data Driven Control and Learning Systems Conference (DDCLS), 2019: 277-281.
[20]	HUANG G, LIU Z, Van Der Maaten L, et al. Densely connected convolutional networks[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2017: 4700-4708.
[21]	Pech-Pacheco J L, Cristóbal G, Chamorro-Martinez J, et al. Diatom autofocusing in bright field microscopy: a comparative study[C]//Proceedings 15th International Conference on Pattern Recognition, 2000, 3: 314-317.
[22]	JIANG Q, LIU Y, YAN Y, et al. A contour angle orientation for power equipment infrared and visible image registration[J]. IEEE Transactions on Power Delivery, 2020, 36(4): 2559-2569. http://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=9149795
[23]	DENG J, DONG W, Socher R, et al. Imagenet: A large-scale hierarchical image database[C]//2009 IEEE Conference on Computer Vision and Pattern Recognition, 2009: 248-255.