航拍电力杆塔图像中销钉唯一性识别研究

doi:10.19595/j.cnki.1000-6753.tces.230760

摘要
图/表
参考文献
相关文章 (1)

全文: PDF (5224 KB)
输出: BibTeX | EndNote (RIS)

摘要

近年来,基于深度学习的缺陷销钉检测技术在电力杆塔巡检领域得到广泛应用。但是该技术只能在图像空间对缺陷销钉进行定位,无法准确给出缺陷销钉在实际电力杆塔中对应销钉的唯一编号。针对该问题,该文提出一种融合深度学习与三维重投影的销钉唯一性识别方法。首先通过构建巡检电力杆塔系统的三维模型,进行电力杆塔系统中部件模型的编号获取以及部件模型三维空间包围盒顶点和中心点坐标的计算。然后利用YOLOv5模型在图像空间对销钉及上下文相关目标进行检测。再通过该文提出的融合结构约束的相机位姿估计算法对航拍图像的相机位姿进行估计。最后将三维电力杆塔系统中的销钉包围盒按相机位姿进行重投影与销钉检测框进行匹配度计算,在图像空间中输出带有具体编号的销钉检测框。在YOLOv5模型输出的销钉检测框基础上进行销钉唯一性识别,利用仿真数据进行实验得到的唯一性识别正确率为100%,利用实拍数据进行实验得到的唯一性识别正确率为93.1%,验证了该文提出的销钉唯一性识别方法的有效性。通过对缺陷销钉进行唯一性识别,可以减少人工对故障图像的二次审核工作量,准确统计缺陷销钉的数量及具体位置信息,为后续故障维修及故障相关性分析等精细化管理提供支持。

	服务

	把本文推荐给朋友
	加入我的书架
	加入引用管理器
	E-mail Alert
	RSS
	作者相关文章
	阎光伟
	马颐琳
	焦润海
	何慧

关键词 ：电力巡检, 销钉检测, 相机位姿估计, 唯一性识别

Abstract：

With the development of artificial intelligence technology, defect pin detection technology based on deep learning has been widely applied in the field of power tower inspection. However, this technology can only locate the defect pin in the image space, but cannot provide a unique number for the defect pin in the actual power tower, and cannot automatically determine the actual position of the defect pin. Moreover, for multi-angle images of the same part, the defect pin detection algorithm using depth learning may repeatedly label the same defect pin, which may cause repeated statistics of defect pins.
To address this issue, this paper proposes a pin uniqueness identification method that integrates deep learning and 3D reprojection, which is essentially an extension of the 2D object detection task. Firstly, a three-dimensional model of the inspection power tower system is constructed to obtain vertices and center points of the components 3D space bounding box, and unique number of the components in the power tower system. Then, the YOLOv5 model is used to detect pins and contextual sensitive targets in the image space. The contextual targets include a group of large-scale hardware, a mesoscale specific hardware, and a small-scale bolt head. Filter detection results that are not within the inspection area by using the hardware target group bounding box. Then use the camera pose estimation algorithm that integrates structural constraints proposed in this article to estimate the camera pose of the image. Use the center points of the pins and bolt heads 2D bounding boxes as 2D feature points, and the center point of the pins and bolt heads 3D space bounding boxes as 3D feature point to form point-to-point constraints. Use P3P algorithm for camera pose estimation. For the unknown 2D-3D correspondence, four 2D feature points are selected by using the specific hardware bounding box, and the matching solution space of 2D-3D feature points is constructed by enumeration method. Calculate the camera pose for each solution separately, and use the deviation error of reprojection to choose the optimal pose. Finally, the numbered pin 3D space bounding box will be reprojected according to the optimal pose, and the matching degree will be calculated with the pin bounding box. The number of the reprojection box with the highest matching degree will be selected as the number of the pin bounding box, and the pin bounding box with unique number will be output in the image space.
In addition, the article also proposes a method for generating multi-pose simulation images and an label automatic generation technology based on occlusion detection, which is used to achieve automatic construction of simulation image datasets. The pin uniqueness identification accuracy of the evaluation index in this article is calculated based on the YOLOv5 model detection results. Experiments on simulated images gets 100% pin uniqueness identification accuracy, while experiments on real images gets 93.1%. By defect pin uniqueness identification, the bounding box with unique number of pins are output in the image space, and automatically count the total and specific position of defect pins in the inspection area based on the corresponding relationship between the number and position information, it helps inspection personnel quickly identify defect pins in multiple inspection images of the same inspection area, and provides assistance for refined management such as fault maintenance and fault correlation analysis.

Key words： Power inspection pin detection camera pose estimation uniqueness identification

收稿日期: 2023-05-25

PACS:

TP181

基金资助:

国家自然科学基金资助项目（62073133）

通讯作者: 阎光伟 1971年生,男,副教授,硕士生导师,研究方向为计算机图形/图像技术在电力系统中的应用。E-mail：yan_guang_wei@126.com

作者简介: 马颐琳 1999年生,女,硕士研究生,主要研究方向为计算机图形/图像技术在电力系统中的应用。E-mail：120212227125@ncepu.edu.cn.

引用本文:

阎光伟, 马颐琳, 焦润海, 何慧. 航拍电力杆塔图像中销钉唯一性识别研究[J]. 电工技术学报, 0, (): 239634-239634. Yan Guangwei, Ma Yilin, Jiao Runhai, He Hui. Research on Pin Uniqueness Identification in Aerial Power Tower Images. Transactions of China Electrotechnical Society, 0, (): 239634-239634.

链接本文:

https://dgjsxb.ces-transaction.com/CN/10.19595/j.cnki.1000-6753.tces.230760 https://dgjsxb.ces-transaction.com/CN/Y0/V/I/239634

[1] 郑含博, 李金恒, 刘洋, 等. 基于改进YOLOv3的电力设备红外目标检测模型[J]. 电工技术学报, 2021, 36(7): 1389-1398.
Zheng Hanbo, Li Jinheng, Liu Yang, et al.Infrared object detection model for power equipment based on improved YOLOv3[J]. Transactions of China Electrotechnical Society, 2021, 36(7): 1389-1398.
[2] 王卓, 王玉静, 王庆岩, 等. 基于协同深度学习的二阶段绝缘子故障检测方法[J]. 电工技术学报, 2021, 36(17): 3594-3604.
Wang Zhuo, Wang Yujing, Wang Qingyan, et al.Two stage insulator fault detection method based on collaborative deep learning[J]. Transactions of China Electrotechnical Society, 2021, 36(17): 3594-3604.
[3] 徐建军, 黄立达, 闫丽梅, 等. 基于层次多任务深度学习的绝缘子自爆缺陷检测[J]. 电工技术学报, 2021, 36(7): 1407-1415.
Xu Jianjun, Huang Lida, Yan Limei, et al.Insulator self-explosion defect detection based on hierarchical multi-task deep learning[J]. Transactions of China Electrotechnical Society, 2021, 36(7): 1407-1415.
[4] 宋立业, 刘帅, 王凯, 等. 基于改进EfficientDet的电网元件及缺陷识别方法[J]. 电工技术学报, 2022, 37(9): 2241-2251.
Song Liye, Liu Shuai, Wang Kai, et al.Identification method of power grid components and defects based on improved EfficientDet[J]. Transactions of China Electrotechnical Society, 2022, 37(9): 2241-2251.
[5] 王建, 吴昊, 张博, 等. 不平衡样本下基于迁移学习-AlexNet的输电线路故障辨识方法[J]. 电力系统自动化, 2022, 46(22): 182-191.
Wang Jian, Wu Hao, Zhang Bo, et al.Fault identification method for transmission line based on transfer learning-AlexNet with imbalanced samples[J]. Automation of Electric Power Systems, 2022, 46(22): 182-191.
[6] 李雪峰, 刘海莹, 刘高华, 等. 基于深度学习的输电线路销钉缺陷检测[J]. 电网技术, 2021, 45(8): 2988-2995.
Li Xuefeng, Liu Haiying, Liu Gaohua, et al.Transmission line pin defect detection based on deep learning[J]. Power System Technology, 2021, 45(8): 2988-2995.
[7] 李瑞生, 张彦龙, 翟登辉, 等. 基于改进SSD的输电线路销钉缺陷检测[J]. 高电压技术, 2021, 47(11): 3795-3802.
Li Ruisheng, Zhang Yanlong, Zhai Denghui, et al.Pin defect detection of transmission line based on improved SSD[J]. High Voltage Engineering, 2021, 47(11): 3795-3802.
[8] 王红星, 翟学锋, 陈玉权, 等. 基于改进Cascade R-CNN的两阶段销钉缺陷检测模型[J]. 科学技术与工程, 2021, 21(15): 6373-6379.
Wang Hongxing, Zhai Xuefeng, Chen Yuquan, et al.Two-stage pin defect detection model based on improved cascade R-CNN[J]. Science Technology and Engineering, 2021, 21(15): 6373-6379.
[9] Jiao Runhai, Liu Yanzhi, He Hui, et al.A deep learning model for small-size defective components detection in power transmission tower[J]. IEEE Transactions on Power Delivery, 2022, 37(4): 2551-2561.
[10] 戚银城, 武学良, 赵振兵, 等. 嵌入双注意力机制的Faster R-CNN航拍输电线路螺栓缺陷检测[J]. 中国图象图形学报, 2021, 26(11): 2594-2604.
Qi Yincheng, Wu Xueliang, Zhao Zhenbing, et al.Bolt defect detection for aerial transmission lines using Faster R-CNN with an embedded dual attention mechanism[J]. Journal of Image and Graphics, 2021, 26(11): 2594-2604.
[11] 文师华. 输电线路防振锤复位机器人目标识别及定位方法研究[D]. 沈阳: 沈阳航空航天大学, 2020.
[12] 胡国雄. 输电线路航拍图像目标检测算法研究[D]. 南京: 南京航空航天大学, 2019.
[13] 陈鑫. 基于无人机航拍的电力线故障定位算法研究[D]. 青岛: 山东科技大学, 2018.
[14] 高伟, 何搏洋, 张婷, 等. 基于注意力机制的变电站作业场景三维目标检测[J]. 激光与光电子学进展, 2022, 59(22): 165-173.
Gao Wei, He Boyang, Zhang Ting, et al.Three-dimensional object detection in substation operation scene based on attention mechanism[J]. Laser & Optoelectronics Progress, 2022, 59(22): 165-173.
[15] 张婷, 张兴忠, 王慧民, 等. 基于图神经网络的变电站场景三维目标检测[J]. 计算机工程与应用, 2023, 59(9): 329-336.
Zhang Ting, Zhang Xingzhong, Wang Huimin, et al.3D object detection in substation scene based on graph neural network[J]. Computer Engineering and Applications, 2023, 59(9): 329-336.
[16] Glenn J, Alex S and Jirka B. YOLOv5[CP/OL]. (2022-02-22).[2023-03-25]. https://github.com/ultralytics/yolov5
[17] 周佳乐, 朱兵, 吴芝路. 融合二维图像和三维点云的相机位姿估计[J]. 光学精密工程, 2022, 30(22): 2901-2912.
Zhou Jiale, Zhu Bing, Wu Zhilu.Camera pose estimation based on 2D image and 3D point cloud fusion[J]. Optics and Precision Engineering, 2022, 30(22): 2901-2912.
[18] 王平, 付辉, 徐贵力. 基于旋转搜索的相机位姿估计和对应点匹配[J]. 航空学报, 2023, 44(2): 296-307.
Wang Ping, Fu Hui, Xu Guili.Camera pose estimation and corresponding points matching based on rotation search[J]. Acta Aeronautica et Astronautica Sinica, 2023, 44(2): 296-307.
[19] Ke Tong, Roumeliotis S I.An efficient algebraic solution to the perspective-three-point problem[C]//2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Honolulu, HI, USA, 2017: 4618-4626.
[20] 仲林林, 胡霞, 刘柯妤. 基于改进生成对抗网络的无人机电力杆塔巡检图像异常检测[J]. 电工技术学报, 2022, 37(9): 2230-2240, 2262.
Zhong Linlin, Hu Xia, Liu Keyu.Power tower anomaly detection from unmanned aerial vehicles inspection images based on improved generative adversarial network[J]. Transactions of China Electrotechnical Society, 2022, 37(9): 2230-2240, 2262.