Infrared and Visible Image Matching Method for Power Equipment Based on Local and Global Features
Feng Xugang1,2,3, Ruan Shanhui3, Wang Zhengbing1,2,3, An Shuo3, Zhang Keqi3
1. Anhui Province Engineering Laboratory of Intelligent Demolition Equipment Anhui University of Technology Maanshan 243032 China; 2. Anhui Engineering Research Center for Intelligent Applications and Security of Industrial Internet Anhui University of Technology Maanshan 243032 China; 3. School of Electrical and Information Engineering Anhui University of Technology Maanshan 243032 China
Abstract：The most intuitive manifestation of power equipment failure is temperature abnormality, through the fusion of infrared and visible images of power equipment can realize the analysis and detection of equipment operation status. Aiming at the problem that the matching process of infrared and visible images of power equipment is greatly affected by the local intensity difference of the images and the difficulty of feature point description and matching, an infrared and visible image matching method for power equipment based on local and global features is proposed. First, the infrared and visible images of power equipment undergo preprocessing through grayscale conversion and normalization. The Canny algorithm is then applied to extract prominent contour features. Secondly, a threshold is set to solve the problem of possible misestimation in k-cosine estimation of curvature. Then the feature points in the contour image are detected separately by the multi-scale corner detection algorithm based on arithmetic mean k-cosine curvature. The proposed curvature adaptive weighting-based principal direction assignment method is used to assign feature principal directions to each feature point to achieve scale and rotation invariance. Then the PIIFD and global context descriptor are constructed for each feature point, and the proposed similarity bidirectional matching method is utilized to complete the preliminary matching. Finally, the RANSAC algorithm is used to obtain the final matching results and thus the parameters of the affine transformation model between images. The experimental results show that the algorithm in this paper successfully matches 10 groups of images, and the average accuracy of the proposed method is 93.41%, and the average repetition rate is 33.56%, which verifies that the matching method in this paper can effectively match the infrared and visible images of power equipment. Compared with PIIFD, LGHD and CAO matching algorithms, the number of correct matching points is significantly increased, the average accuracy is improved by 50.71, 27.62 and 11.11 percentage points compared with the other three algorithms, and the average repetition rate is improved by 27.69, 28.81 and 19.18 percentage points respectively. The accurate matching with visible images can still be realized after scaling and rotating changes to the infrared images respectively. The contributions of the proposed infrared and visible image matching method for power equipment based on local and global features are as follows: (1) The improved k-cosine curvature can accurately reflect the curvature meaning, and a large number of feature points with high accuracy and scale invariance can be extracted by multi-scale arithmetic mean curvature feature detection. (2) The principal orientation assignment method based on curvature adaptive weighting proposed in this paper is robust when small-angle rotations occur between image pairs. (3) By proposing a feature description method that combines global context information with local feature information, the effect of local intensity differences on the matching results is effectively overcome, and infrared and visible image matching with scale invariance is realized.
冯旭刚, 阮善会, 王正兵, 安硕, 张科琪. 基于局部和全局特征的电力设备红外和可见光图像匹配方法[J]. 电工技术学报, 2025, 40(7): 2236-2246.
Feng Xugang, Ruan Shanhui, Wang Zhengbing, An Shuo, Zhang Keqi. Infrared and Visible Image Matching Method for Power Equipment Based on Local and Global Features. Transactions of China Electrotechnical Society, 2025, 40(7): 2236-2246.
[1] 韩翔宇, 纽春萍, 何海龙, 等. 电磁式断路器状态监测与智能评估技术综述[J]. 电工技术学报, 2023, 38(8): 2191-2210. Han Xiangyu, Niu Chunping, He Hailong, et al.Review of condition monitoring and intelligent assessment of electromagnetic circuit breaker[J]. Transactions of China Electrotechnical Society, 2023, 38(8): 2191-2210. [2] 苟军年, 杜愫愫, 刘力. 基于改进掩膜区域卷积神经网络的输电线路绝缘子自爆检测[J]. 电工技术学报, 2023, 38(1): 47-59. Gou Junnian, Du Susu, Liu Li.Transmission line insulator self-explosion detection based on improved mask region-convolutional neural network[J]. Transactions of China Electrotechnical Society, 2023, 38(1): 47-59. [3] 杨帆, 王梦珺, 谭天, 等. 基于目标像素宽度识别的电力设备红外成像单目测距改进算法[J]. 电工技术学报, 2023, 38(8): 2244-2254. Yang Fan, Wang Mengjun, Tan Tian, et al.An improved monocular ranging method for infrared image of power equipment based on the pixel width recognition of objects[J]. Transactions of China Electrotechnical Society, 2023, 38(8): 2244-2254. [4] 李斌, 屈璐瑶, 朱新山, 等. 基于多尺度特征融合的绝缘子缺陷检测[J]. 电工技术学报, 2023, 38(1): 60-70. Li Bin, Qu Luyao, Zhu Xinshan, et al.Insulator defect detection based on multi-scale feature fusion[J]. Transactions of China Electrotechnical Society, 2023, 38(1): 60-70. [5] 普子恒, 张隆, 余欣芸, 等. 500 kV换流变压器无人机巡检路径规划[J]. 电工技术学报, 2023, 38(增刊1): 204-213. Pu Ziheng, Zhang Long, Yu Xinyun, et al.Route planning of unmanned aerial vehicle patrol inspection for 500 kV converter transformer[J]. Transactions of China Electrotechnical Society, 2023, 38(S1): 204-213. [6] 李浩伟, 张楚岩, 史梓男, 等. 基于红外图像检测和改进YOLOv4的高压套管故障识别方法[J]. 高压电器, 2023, 59(11): 24-31. Li Haowei, Zhang Chuyan, Shi Zinan, et al.Fault identification method of high-voltage bushing based on infrared image detection and improved YOLOv4 algorithm[J]. High Voltage Apparatus, 2023, 59(11): 24-31. [7] Ma Jiayi, Ma Yong, Li Chang.Infrared and visible image fusion methods and applications: a survey[J]. Information Fusion, 2019, 45: 153-178. [8] 李云红, 罗雪敏, 苏雪平, 等. 基于改进曲率尺度空间算法的电力设备红外与可见光图像配准[J]. 激光与光电子学进展, 2022, 59(12): 1210010. Li Yunhong, Luo Xuemin, Su Xueping, et al.Registration method for power equipment infrared and visible images based on improved curvature scale space algorithm[J]. Laser & Optoelectronics Progress, 2022, 59(12): 1210010. [9] 孙浩, 王朋. 一种基于区域划分的改进ORB算法[J]. 北京航空航天大学学报, 2020, 46(9): 1763-1769. Sun Hao, Wang Peng.An improved ORB algorithm based on region division[J]. Journal of Beijing University of Aeronautics and Astronautics, 2020, 46(9): 1763-1769. [10] 唐泽恬, 丁召, 曾瑞敏, 等. 基于改进的Harris和二次归一化互相关的量子图像拼接算法[J]. 激光与光电子学进展, 2020, 57(10): 101005. Tang Zetian, Ding Zhao, Zeng Ruimin, et al.Quantum image stitching algorithm based on improved Harris and quadratic normalized cross correlation[J]. Laser & Optoelectronics Progress, 2020, 57(10): 101005. [11] Sedaghat A, Mokhtarzade M, Ebadi H.Uniform robust scale-invariant feature matching for optical remote sensing images[J]. IEEE Transactions on Geoscience and Remote Sensing, 2011, 49(11): 4516-4527. [12] Bay H, Ess A, Tuytelaars T, et al.Speeded-up robust features (SURF)[J]. Computer Vision and Image Understanding, 2008, 110(3): 346-359. [13] Chen Jian, Tian Jie, Lee N, et al.A partial intensity invariant feature descriptor for multimodal retinal image registration[J]. IEEE Transactions on Bio-Medical Engineering, 2010, 57(7): 1707-1718. [14] Li Ning, Li Yuxuan, Jiao Jichao. Multimodal remote sensing image registration based on adaptive multi-scale PIIFD[J/OL]. Multimedia Tools and Applications, 2024: 1-13[2024-05-16]. https://doi.org/10.1007/s11042-024-18756-1. [15] Sun Wanfeng, Gao Haibo, Li Cheng.A two-stage registration strategy for thermal-visible images in substations[J]. Applied Sciences, 2024, 14(3): 1158. [16] 朱英宏, 李俊山, 汤雨. 基于CSS角点提取的红外与可见光图像匹配算法[J]. 系统工程与电子技术, 2011, 33(11): 2540-2545. Zhu Yinghong, Li Junshan, Tang Yu.Matching algorithm for IR/visible images based on CSS corner extraction[J]. Systems Engineering and Electronics, 2011, 33(11): 2540-2545. [17] 姜骞, 刘亚东, 方健, 等. 基于轮廓特征的电力设备红外和可见光图像配准方法[J]. 仪器仪表学报,2020, 41(11): 252-260. Jiang Qian, Liu Yadong, Fang Jian, et al.Registration method for power equipment infrared and visible images based on contour feature[J]. Chinese Journal of Scientific Instrument, 2020, 41(11): 252-260. [18] Wang Zhengbing, Feng Xugang, Xu Guili, et al.A robust visible and infrared image matching algorithm for power equipment based on phase congruency and scale-invariant feature[J]. Optics and Lasers in Engineering, 2023, 164: 107517. [19] Aguilera C A, Sappa A D, Toledo R.LGHD: a feature descriptor for matching across non-linear intensity variations[C]//2015 IEEE International Conference on Image Processing (ICIP), Quebec City, QC, Canada, 2015: 178-181. [20] Li Jiayuan, Hu Qingwu, Ai Mingyao.RIFT: multi-modal image matching based on radiation-variation insensitive feature transform[J]. IEEE Transactions on Image Processing, 2020, 29: 3296-3310. [21] DeTone D, Malisiewicz T, Rabinovich A. SuperPoint: self-supervised interest point detection and descrip-tion[C]//2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), Salt Lake City, UT, USA, 2018: 224-236. [22] Sarlin P E, DeTone D, Malisiewicz T, et al. SuperGlue: learning feature matching with graph neural networks[C]//2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Seattle, WA, USA, 2020: 4938-4947. [23] Shi Yan, Cai Junxiong, Shavit Y, et al.ClusterGNN: cluster-based coarse-to-fine graph neural network for efficient feature matching[C]//2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), New Orleans, LA, USA, 2022: 12517-12526. [24] Zhang Shizheng, Li Baohuan, Zhang Zhifeng, et al.Robust corner finding based on multi-scale K-cosine angle detection[J]. IEEE Access, 2020, 8: 66741-66748. [25] Zhong Baojiang, Liao Wenhe.Direct curvature scale space: theory and corner detection[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2007, 29(3): 508-512. [26] Li Dongqing, Zhong Baojiang, Ma Kaikuang.Multi-scale corner detection based on arithmetic mean curvature[C]//2015 IEEE China Summit and Inter-national Conference on Signal and Information Processing (ChinaSIP), Chengdu, China, 2015: 433-437. [27] Rosenfeld A, Johnston E. Angle detection on digital curves[J]. IEEE Transactions on Computers, 1973, C-22(9): 875-878. [28] Wang Zhengbing, Feng Xugang, Li Dan, et al.A robust multi-source image matching method for power equipment based on improved PIIFD[C]//2023 8th International Conference on Signal and Image Processing (ICSIP), Wuxi, China, 2023: 254-258. [29] Gao Chenzhong, Li Wei.Multi-scale PIIFD for registration of multi-source remote sensing images[J]. Journal of Beijing Institute of Technology, 2021, 30(2): 113-124. [30] 谭建豪, 殷旺, 刘力铭, 等. 引入全局上下文特征模块的DenseNet孪生网络目标跟踪[J]. 电子与信息学报, 2021, 43(1): 179-186. Tan Jianhao, Yin Wang, Liu Liming, et al.DenseNet-Siamese network with global context feature module for object tracking[J]. Journal of Electronics & Information Technology, 2021, 43(1): 179-186. [31] Deng Haowen, Birdal T, Ilic S.PPFNet: global context aware local features for robust 3D point matching[C]// 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, 2018: 195-205. [32] 朱惠玲, 牛哲文, 黄克灿, 等. 基于单阶段目标检测算法的变电设备红外图像目标识别及定位[J]. 电力自动化设备, 2021, 41(8): 217-224. Zhu Huiling, Niu Zhewen, Huang Kecan, et al.Identification and location of infrared image for substation equipment based on single-stage object detection algorithm[J]. Electric Power Automation Equipment, 2021, 41(8): 217-224.
2025, Vol. 40 
