风电场无人机巡检红外叶片图像拼接算法

周登科; 郭星辰; 史凯特; 汤鹏; 郑开元; 马鹏阁

风电场无人机巡检红外叶片图像拼接算法

1.
中国长江三峡集团有限公司科学技术研究院, 北京 100038
2.
郑州航空工业管理学院智能工程学院, 河南郑州 450015

基金项目:

国家自然科学基金民航联合基金重点项目 U1833203

中国长江三峡集团有限公司企业科研项目“海上风电无人机光电巡检系统技术研究及应用示范” 202003035

河南省科技攻关 202102210330

河南省科技攻关 222102210136

郑州航空工业管理学院研究生教育创新计划基金 2021CX48

详细信息

作者简介:
周登科（1995-），男，四川人，初级工程师，硕士研究生，从事图像处理工作。E-mail：zhou_dengke@ctg.com.cn

通讯作者:
郭星辰（1997-），男，河南郑州人，硕士研究生，从事图像处理研究。E-mail：478973623@qq.com

中图分类号: TP391
计量
- 文章访问数: 159
- HTML全文浏览量: 47
- PDF下载量: 36
出版历程
- 收稿日期: 2022-09-25
- 修回日期: 2022-12-12
- 刊出日期: 2023-11-19

Infrared Blade Image Stitching Algorithm for Wind Farm UAV Inspection

1.
China Three Gorges Corporation Research Institute, Beijing 100038, China
2.
Zhengzhou University of Aeronautics, Zhengzhou 450015, China

摘要

摘要: 针对无人机拍摄叶片红外图像背景冗余信息较多、拼接精度不高等问题，本文提出一种基于形态学改进Chan-Vese分割与局部特征匹配的红外风机叶片图像拼接算法，首先，对图像进行中值滤波降噪，使用形态学运算改进基于Chan-Vese模型的水平集算法，生成表达主体的掩膜。基于掩膜去除冗余背景提取局部Harris特征点；对掩膜进行二次形态学腐蚀处理，抑制边界锯齿像素上的伪特征点；最后，使用暴力匹配及随机抽样一致（Random Sample Consensus, RANSAC）算法筛选出有效匹配点对，计算单应性矩阵实现匹配拼接。与传统图像分割下Harris拼接算法相比，本文改进后的算法拼接精度有明显提高，在不同的测试场景下显示出较强鲁棒性。
- 风力发电机叶片 /
- 红外图像拼接 /
- 水平集 /
- 形态学运算 /
- Harris特征点
Abstract: Aiming at more redundant background information and low stitching accuracy of the infrared images of the blades taken by UAV (Unmanned Aerial Vehicle), In this study, we proposed a stitching algorithm for infrared wind turbine blade images combining the Chan-Vese model and morphology. First, we subjected the image to median filtering and noise reduction, and a morphological operation improved a level-set algorithm based on the Chan-Vese model to generate the mask of the expression subject. We extracted Harris feature points by removing redundant backgrounds based on the mask. We performed morphological etching on the mask to suppress the pseudo-feature points on the boundary-jagged pixels. We used violent matching and the RANSAC algorithm to screen out effective matching point pairs and calculate the homography matrix to realize matching and splicing. Compared with the Harris stitching algorithm under traditional image segmentation, the stitching accuracy of the improved algorithm significantly improved, and it showed strong robustness in different test scenarios.
- wind turbine blades /
- infrared image stitching /
- level set /
- morphological operation /
- Harris feature point

HTML全文

图 1 海上风机

Figure 1. An Offshore wind turbine

下载: 全尺寸图片幻灯片

图 2 无人机红外图像采集及叶片拼接系统流程

Figure 2. UAV infrared image acquisition and blade Mosaic system process

下载: 全尺寸图片幻灯片

图 3 不同迭代次数演化结果

Figure 3. Evolutionary results for different times of iteration

下载: 全尺寸图片幻灯片

图 4 CV模型下目标分割

Figure 4. Target segmentation under CV model

下载: 全尺寸图片幻灯片

图 5 连通域标识方式

Figure 5. Connected domain identifier

下载: 全尺寸图片幻灯片

图 6 连通域标记

Figure 6. Connected domain label

下载: 全尺寸图片幻灯片

图 7 基于连通域估计滤波结果

Figure 7. The filtering results are estimated based on the connected domain

下载: 全尺寸图片幻灯片

图 8 形态学处理后效果

Figure 8. Effect after morphological treatment

下载: 全尺寸图片幻灯片

图 9 背景去除后特征点显示结果

Figure 9. Feature point display after background removal

下载: 全尺寸图片幻灯片

图 10 RANSAC算法筛选结果

Figure 10. RANSAC algorithm matching

下载: 全尺寸图片幻灯片

图 11 形态学掩膜约束后匹配结果

Figure 11. Morphological mask constraints

下载: 全尺寸图片幻灯片

图 12 拼接使用的部分样本数据

Figure 12. The sample data used for stitching

下载: 全尺寸图片幻灯片

图 13 图片拼接效果对比

Figure 13. Comparison of image stitching effects

下载: 全尺寸图片幻灯片

图 14 风力发电机叶片图像拼接整体效果

Figure 14. Overall effect of wind turbine blade picture stitching

下载: 全尺寸图片幻灯片

表 1 4种算法RMSE值

Table 1 RMSE values of four algorithms

Group	Ostu -Harris	IT -Harris	CV -Harris	Ours
1	230.054	239.641	112.521	27.376
2	179.600	180.267	129.603	11.117
3	123.521	117.466	115.621	25.009
4	68.046	87.921	87.761	66.747
5	39.196	34.262	35.067	30.415
6	17.463	25.906	18.391	16.821

下载: 导出CSV

表 2 4种算法时间消耗

Table 2 Time consumption of the 4 algorithms s

Group	Ostu-Harris	IT-Harris	CV-Harris	Ours
1	3.661	2.896	10.337	10.503
2	3.145	2.899	10.365	10.178
3	3.025	2.779	10.639	10.638
4	3.002	3.02	10.437	10.381
5	3.269	3.051	10.635	11.016
6	3.064	2.956	10.521	10.594
Average	3.194	2.934	10.489	10.551

下载: 导出CSV

参考文献(14)

[1]	李禹桥. 基于旋翼无人机的风电叶片自主巡检系统研究[D]. 徐州: 中国矿业大学, 2021. LI Yuqiao. Research on Wind Turbine Blade Autonomous Inspection System based on Rotorcraft UAV[D]. Xuzhou: China University of Mining and Technology, 2021.
[2]	张明辉. 风力发电机故障检修与处理[J]. 科技创新导报, 2019, 16(10): 132-133. https://www.cnki.com.cn/Article/CJFDTOTAL-ZXDB201910075.htm ZHANG Minghui. Wind turbine fault repair and treatment[J]. Science and Technology Innovation Herald, 2019, 16(10): 132-133. https://www.cnki.com.cn/Article/CJFDTOTAL-ZXDB201910075.htm
[3]	王蔚, 刘丹. 我国风电市场: 前景一路看好[N]. 经济参考报, [2006-06-15]. WANG Wei, LIU Dan. China's wind power market: prospects all the way bright[N]. Economic Information Daily, [2006-06-15].
[4]	王浩, 闫号, 叶海瑞, 等. 基于无人机的光伏电站智能巡检[J]. 红外技术, 2022, 44(5): 537-542. http://hwjs.nvir.cn/article/id/9b100b0a-485a-47dd-bc1a-357a9bc0091f WANG Hao, YAN Hao, YE Hairui, et al. Intelligent patrol inspection of photovoltaic power station based on UAVs[J]. Infrared Technology, 2022, 44(5): 537-542. http://hwjs.nvir.cn/article/id/9b100b0a-485a-47dd-bc1a-357a9bc0091f
[5]	宫妍, 位冲冲. 图像拼接关键技术研究综述[J]. 电脑知识与技术, 2021, 17(30): 106-108. Doi: 10.14004/j.cnki.ckt.2021.2895. GONG Yan, WEI Chongchong. Review on key techniques of image stitching[J]. Computer Knowledge and Technology, 2021, 17(30): 106-108. Doi: 10.14004/j.cnki.ckt.2021.2895.
[6]	CUI Z, QI W, LIU Y. A fast image template matching algo-rithm based on normalized cross correlation[J]. Journal of Physics Conference Series, 2020, 1693: 012163. DOI: 10.1088/1742-6596/1693/1/012163
[7]	马宝琰, 汤磊, 赵晶, 等. 风电叶片图像直线特征检测与拼接方法[J]. 哈尔滨理工大学学报, 2020, 25(5): 83-92. Doi: 10.15938/j.jhust.2020.05.012. MA Baoyan, TANG Lei, ZHAO Jing, et al. Straight line features detection and mosaic of wind power blades image[J]. Journal of Harbin University of Science and Technology, 2020, 25(5): 83-92. Doi: 10.15938/j.jhust.2020.05.012.
[8]	卢泉, 杨振华, 黄粒峰. 改进最佳缝合线的红外图像拼接方法[J]. 红外技术, 2022, 44(6): 580-586. http://hwjs.nvir.cn/article/id/ff102dc9-5b92-41f9-8a8f-813d8d5f0c8b LU Quan, YANG Zhenhua, HUANG Lifeng. Infrared image mosaic method for improving the best seam-line[J]. Infrared Technology, 2022, 44(6): 580-586. http://hwjs.nvir.cn/article/id/ff102dc9-5b92-41f9-8a8f-813d8d5f0c8b
[9]	傅子秋, 张晓龙, 余成, 等. 多场景下基于快速相机标定的柱面图像拼接方法[J]. 光电工程, 2020, 47(4): 74-86. https://www.cnki.com.cn/Article/CJFDTOTAL-GDGC202004008.htm FU Ziqiu, ZHANG Xiaolong, YU Cheng, et al. Cylindrical image mosaic method based on fast camera calibration in multi-scene[J]. Opto-Electronic Engineering, 2020, 47(4): 74-86. https://www.cnki.com.cn/Article/CJFDTOTAL-GDGC202004008.htm
[10]	何赟泽, 张帆, 刘昊, 等. 风机叶片无人机红外热图像拼接方法[J]. 电子测量与仪器学报, 2022, 36(7): 44-53. Doi: 10.13382/j.jemi.B2105058. HE Yunze, ZHANG Fan, LIU Hao, et al. Infrared image stitch method of wind turbine blade based on UAV[J]. Journal of Electronic Measurement and Instrument, 2022, 36(7): 44-53. Doi: 10.13382/j.jemi.B2105058.
[11]	方喜波. 光电侦察吊舱对海广域搜索方法[J]. 红外技术, 2021, 43(11): 1055-1060. http://hwjs.nvir.cn/article/id/9db773d6-e53b-4486-a8ca-f32834bc9f13 FANG Xibo. Searching method of the wide area of optical recon pod for sea targets[J]. Infrared Technology, 2021, 43(11): 1055-1060. http://hwjs.nvir.cn/article/id/9db773d6-e53b-4486-a8ca-f32834bc9f13
[12]	Osher S, Sethian J A. Fronts propagating with curvature-dependent speed: algorithms based on Hamilton-Jacobi formulations[J]. Journal of Computational Physics, 1988, 79(1): 12-49.
[13]	Ghaili A M, Mashohor S, Ramli A R. Vertical-edge-based car-license-plate detection method[J]. IEEE Transactions on Vehicular Technology, 2013, 62(1): 26-38.
[14]	张见双, 张红民, 罗永涛, 等. 一种改进的Harris角点检测的图像配准方法[J]. 激光与红外, 2017, 47(2): 230-233. https://www.cnki.com.cn/Article/CJFDTOTAL-JGHW201702021.htm ZHANG Jianshuang, ZHANG Hongmin, LUO Yongtao, et al. Image registration method based on improved Harris corner detection algorithm[J]. Laser & Infrared, 2017, 47(2): 230-233. https://www.cnki.com.cn/Article/CJFDTOTAL-JGHW201702021.htm

施引文献(6)

期刊类型引用(3)

1.	MENG Yinjie，WEI Zhengjun，YAN Ziling，WANG Jindong. Development of a bias power supply for Geiger mode avalanche photodiodes. Optoelectronics Letters. 2023(11): 659-665 . 必应学术
2.	陈良洲，鲁猛，陈有林. 基于TPS61175的BOOST升压及电荷泵倍压电路设计. 电子设计工程. 2021(11): 189-193 . 百度学术
3.	刘智恒. 光电二极管在物理中的应用. 家庭生活指南. 2018(09): 70 . 百度学术