Abstract：With the development of artificial intelligence technology, defect pin detection technology based on deep learning has been widely applied in the field of power tower inspection. However, this technology can only locate the defect pin in the image space, but cannot provide a unique number for the defect pin in the actual power tower, and cannot automatically determine the actual position of the defect pin. Moreover, for multi-angle images of the same part, the defect pin detection algorithm using depth learning may repeatedly label the same defect pin, which may cause repeated statistics of defect pins. To address this issue, this paper proposes a pin uniqueness identification method that integrates deep learning and 3D reprojection, which is essentially an extension of the 2D object detection task. Firstly, a three-dimensional model of the inspection power tower system is constructed to obtain vertices and center points of the components 3D space bounding box, and unique number of the components in the power tower system. Then, the YOLOv5 model is used to detect pins and contextual sensitive targets in the image space. The contextual targets include a group of large-scale hardware, a mesoscale specific hardware, and a small-scale bolt head. Filter detection results that are not within the inspection area by using the hardware target group bounding box. Then use the camera pose estimation algorithm that integrates structural constraints proposed in this article to estimate the camera pose of the image. Use the center points of the pins and bolt heads 2D bounding boxes as 2D feature points, and the center point of the pins and bolt heads 3D space bounding boxes as 3D feature point to form point-to-point constraints. Use P3P algorithm for camera pose estimation. For the unknown 2D-3D correspondence, four 2D feature points are selected by using the specific hardware bounding box, and the matching solution space of 2D-3D feature points is constructed by enumeration method. Calculate the camera pose for each solution separately, and use the deviation error of reprojection to choose the optimal pose. Finally, the numbered pin 3D space bounding box will be reprojected according to the optimal pose, and the matching degree will be calculated with the pin bounding box. The number of the reprojection box with the highest matching degree will be selected as the number of the pin bounding box, and the pin bounding box with unique number will be output in the image space. In addition, the article also proposes a method for generating multi-pose simulation images and an label automatic generation technology based on occlusion detection, which is used to achieve automatic construction of simulation image datasets. The pin uniqueness identification accuracy of the evaluation index in this article is calculated based on the YOLOv5 model detection results. Experiments on simulated images gets 100% pin uniqueness identification accuracy, while experiments on real images gets 93.3%. By defect pin uniqueness identification, the bounding box with unique number of pins are output in the image space, and automatically count the total and specific position of defect pins in the inspection area based on the corresponding relationship between the number and position information, it helps inspection personnel quickly identify defect pins in multiple inspection images of the same inspection area, and provides assistance for refined management such as fault maintenance and fault correlation analysis.
阎光伟, 马颐琳, 焦润海, 何慧. 航拍电力杆塔图像中销钉唯一性识别研究[J]. 电工技术学报, 2024, 39(17): 5450-5460.
Yan Guangwei, Ma Yilin, Jiao Runhai, He Hui. Research on Pin Uniqueness Identification in Aerial Power Tower Images. Transactions of China Electrotechnical Society, 2024, 39(17): 5450-5460.
[1] 郑含博, 李金恒, 刘洋, 等. 基于改进YOLOv3的电力设备红外目标检测模型[J]. 电工技术学报, 2021, 36(7): 1389-1398. Zheng Hanbo, Li Jinheng, Liu Yang, et al.Infrared object detection model for power equipment based on improved YOLOv3[J]. Transactions of China Electrotechnical Society, 2021, 36(7): 1389-1398. [2] 王卓, 王玉静, 王庆岩, 等. 基于协同深度学习的二阶段绝缘子故障检测方法[J]. 电工技术学报, 2021, 36(17): 3594-3604. Wang Zhuo, Wang Yujing, Wang Qingyan, et al.Two stage insulator fault detection method based on collaborative deep learning[J]. Transactions of China Electrotechnical Society, 2021, 36(17): 3594-3604. [3] 徐建军, 黄立达, 闫丽梅, 等. 基于层次多任务深度学习的绝缘子自爆缺陷检测[J]. 电工技术学报, 2021, 36(7): 1407-1415. Xu Jianjun, Huang Lida, Yan Limei, et al.Insulator self-explosion defect detection based on hierarchical multi-task deep learning[J]. Transactions of China Electrotechnical Society, 2021, 36(7): 1407-1415. [4] 宋立业, 刘帅, 王凯, 等. 基于改进EfficientDet的电网元件及缺陷识别方法[J]. 电工技术学报, 2022, 37(9): 2241-2251. Song Liye, Liu Shuai, Wang Kai, et al.Identification method of power grid components and defects based on improved EfficientDet[J]. Transactions of China Electrotechnical Society, 2022, 37(9): 2241-2251. [5] 王建, 吴昊, 张博, 等. 不平衡样本下基于迁移学习-AlexNet的输电线路故障辨识方法[J]. 电力系统自动化, 2022, 46(22): 182-191. Wang Jian, Wu Hao, Zhang Bo, et al.Fault identification method for transmission line based on transfer learning-AlexNet with imbalanced samples[J]. Automation of Electric Power Systems, 2022, 46(22): 182-191. [6] 李雪峰, 刘海莹, 刘高华, 等. 基于深度学习的输电线路销钉缺陷检测[J]. 电网技术, 2021, 45(8): 2988-2995. Li Xuefeng, Liu Haiying, Liu Gaohua, et al.Transmission line pin defect detection based on deep learning[J]. Power System Technology, 2021, 45(8): 2988-2995. [7] 李瑞生, 张彦龙, 翟登辉, 等. 基于改进SSD的输电线路销钉缺陷检测[J]. 高电压技术, 2021, 47(11): 3795-3802. Li Ruisheng, Zhang Yanlong, Zhai Denghui, et al.Pin defect detection of transmission line based on improved SSD[J]. High Voltage Engineering, 2021, 47(11): 3795-3802. [8] 王红星, 翟学锋, 陈玉权, 等. 基于改进Cascade R-CNN的两阶段销钉缺陷检测模型[J]. 科学技术与工程, 2021, 21(15): 6373-6379. Wang Hongxing, Zhai Xuefeng, Chen Yuquan, et al.Two-stage pin defect detection model based on improved cascade R-CNN[J]. Science Technology and Engineering, 2021, 21(15): 6373-6379. [9] Jiao Runhai, Liu Yanzhi, He Hui, et al.A deep learning model for small-size defective components detection in power transmission tower[J]. IEEE Transactions on Power Delivery, 2022, 37(4): 2551-2561. [10] 戚银城, 武学良, 赵振兵, 等. 嵌入双注意力机制的Faster R-CNN航拍输电线路螺栓缺陷检测[J]. 中国图象图形学报, 2021, 26(11): 2594-2604. Qi Yincheng, Wu Xueliang, Zhao Zhenbing, et al.Bolt defect detection for aerial transmission lines using Faster R-CNN with an embedded dual attention mechanism[J]. Journal of Image and Graphics, 2021, 26(11): 2594-2604. [11] 文师华. 输电线路防振锤复位机器人目标识别及定位方法研究[D]. 沈阳: 沈阳航空航天大学, 2020. [12] 胡国雄. 输电线路航拍图像目标检测算法研究[D]. 南京: 南京航空航天大学, 2019. [13] 陈鑫. 基于无人机航拍的电力线故障定位算法研究[D]. 青岛: 山东科技大学, 2018. [14] 高伟, 何搏洋, 张婷, 等. 基于注意力机制的变电站作业场景三维目标检测[J]. 激光与光电子学进展, 2022, 59(22): 165-173. Gao Wei, He Boyang, Zhang Ting, et al.Three-dimensional object detection in substation operation scene based on attention mechanism[J]. Laser & Optoelectronics Progress, 2022, 59(22): 165-173. [15] 张婷, 张兴忠, 王慧民, 等. 基于图神经网络的变电站场景三维目标检测[J]. 计算机工程与应用, 2023, 59(9): 329-336. Zhang Ting, Zhang Xingzhong, Wang Huimin, et al.3D object detection in substation scene based on graph neural network[J]. Computer Engineering and Applications, 2023, 59(9): 329-336. [16] Glenn J, Alex S, Jirka B. YOLOv5[CP/OL]. (2022-02-22). [2023-03-25]. https://github.com/ultralytics/yolov5 [17] 周佳乐, 朱兵, 吴芝路. 融合二维图像和三维点云的相机位姿估计[J]. 光学精密工程, 2022, 30(22): 2901-2912. Zhou Jiale, Zhu Bing, Wu Zhilu.Camera pose estimation based on 2D image and 3D point cloud fusion[J]. Optics and Precision Engineering, 2022, 30(22): 2901-2912. [18] 王平, 付辉, 徐贵力. 基于旋转搜索的相机位姿估计和对应点匹配[J]. 航空学报, 2023, 44(2): 296-307. Wang Ping, Fu Hui, Xu Guili.Camera pose estimation and corresponding points matching based on rotation search[J]. Acta Aeronautica et Astronautica Sinica, 2023, 44(2): 296-307. [19] Ke Tong, Roumeliotis S I.An efficient algebraic solution to the perspective-three-point problem[C]// 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Honolulu, HI, USA, 2017: 4618-4626. [20] 仲林林, 胡霞, 刘柯妤. 基于改进生成对抗网络的无人机电力杆塔巡检图像异常检测[J]. 电工技术学报, 2022, 37(9): 2230-2240, 2262. Zhong Linlin, Hu Xia, Liu Keyu.Power tower anomaly detection from unmanned aerial vehicles inspection images based on improved generative adversarial network[J]. Transactions of China Electrotechnical Society, 2022, 37(9): 2230-2240, 2262.
2024, Vol. 39 
