基于局部和全局特征的电力设备红外和可见光图像匹配方法

doi:10.19595/j.cnki.1000-6753.tces.240529

摘要
图/表
参考文献
相关文章 (10)

全文: PDF (2534 KB)
输出: BibTeX | EndNote (RIS)

摘要

针对电力设备红外和可见光图像匹配过程受图像局部灰度差异影响大、特征点描述和匹配困难的问题,提出了基于局部和全局特征的电力设备红外和可见光图像匹配方法。首先,利用多尺度角检测算法分别检测红外和可见光图像中的特征点,再使用不同尺度的曲率信息为每个特征点分配特征主方向;其次,分别构建每个特征点的部分灰度不变特征描述符（PIIFD）和全局上下文特征描述符;然后,将两种特征描述符的相似度进行加权融合,并利用双向匹配方法和随机抽样一致（RANSAC）方法筛选出正确的匹配点对;最后,得到图像间的仿射变换模型参数。实验结果表明：该文匹配方法与PIIFD、LGHD和CAO匹配算法相比正确匹配点数目显著增加,平均准确率较其他三种算法分别提高了50.71%、27.62%、11.11%,平均重复度分别提高了27.69%、28.81%、19.18%。

	服务

	把本文推荐给朋友
	加入我的书架
	加入引用管理器
	E-mail Alert
	RSS
	作者相关文章
	冯旭刚
	阮善会
	王正兵
	安硕
	张科琪

关键词 ：电力设备, 图像匹配, 红外和可见光图像, 全局上下文描述符, 特征相似度匹配

Abstract：

The most intuitive manifestation of power equipment failure is temperature abnormality, through the fusion of infrared and visible light images of power equipment can realize the analysis and detection of equipment operation status. Aiming at the problem that the matching process of infrared and visible images of power equipment is greatly affected by the local intensity difference of the images and the difficulty of feature point description and matching, an infrared and visible image matching method for power equipment based on local and global features is proposed.
First, the infrared and visible images of power equipment undergo preprocessing through grayscale conversion and normalization. The Canny algorithm is then applied to extract prominent contour features. Secondly, a threshold is set to solve the problem of possible misestimation in k-cosine estimation of curvature. Then the feature points in the contour image are detected separately by the multi-scale corner detection algorithm based on arithmetic mean k-cosine curvature. The proposed curvature adaptive weighting-based principal direction assignment method is used to assign feature principal directions to each feature point to achieve scale and rotation invariance. Then the PIIFD descriptor and global context descriptor are constructed for each feature point, and the proposed similarity bidirectional matching method is utilized to complete the preliminary matching. Finally, the RANSAC algorithm is used to obtain the final matching results and thus the parameters of the affine transformation model between images.
The experimental results show that the algorithm in this paper successfully matches 10 groups of images, and the average accuracy of the proposed method is 93.41%, and the average repetition rate is 33.56%, which verifies that the matching method in this paper can effectively match the infrared and visible images of power equipment. Compared with PIIFD, LGHD and CAO matching algorithms, the number of correct matching points is significantly increased, the average accuracy is improved by 50.71%, 27.62% and 11.11% compared with the other three algorithms, and the average repetition rate is improved by 27.69%, 28.81% and 19.18% respectively. The accurate matching with visible images can still be realized after scaling and rotating changes to the infrared images respectively.
The contributions of the proposed infrared and visible image matching method for power equipment based on local and global features are as follows: (1) The improved k-cosine curvature can accurately reflect the curvature meaning, and a large number of feature points with high accuracy and scale invariance can be extracted by multi-scale arithmetic mean curvature feature detection. (2) The principal orientation assignment method based on curvature adaptive weighting proposed in this paper is robust when small-angle rotations occur between image pairs. (3) By proposing a feature description method that combines global context information with local feature information, the effect of local intensity differences on the matching results is effectively overcome, and infrared and visible image matching with scale invariance is realized.

Key words： Power equipment image matching infrared and visible images global context descriptor feature similarity matching

收稿日期: 2024-04-02

PACS:

TP391.4

基金资助:

安徽省高等学校自然科学研究重点项目（2023AH051107）、安徽省智能破拆装备工程实验室开放基金项目（APELIDE2022A001）、安徽省工业互联网智能应用与安全工程研究中心开放基金项目（IASII22-07）和安徽高校自然科学研究重大项目（KJ2021ZD0042）资助

通讯作者: 王正兵男,1989年生,博士,硕士生导师,研究方向为工业视觉检测和图像处理。E-mail：wangzb89@ahut.edu.cn

作者简介: 冯旭刚男,1979年生,教授,硕士生导师,研究方向为流程工业先进检测技术及智能优化控制。E-mail：fxg@ahut.edu.cn

引用本文:

冯旭刚, 阮善会, 王正兵, 安硕, 张科琪. 基于局部和全局特征的电力设备红外和可见光图像匹配方法[J]. 电工技术学报, 0, (): 2492930-2492930. Feng Xugang, Ruan Shanhui, Wang Zhengbing, An Shuo, Zhang Keqi. Infrared and Visible Image Matching Method for Power Equipment Based on Local and Global Features. Transactions of China Electrotechnical Society, 0, (): 2492930-2492930.

链接本文:

https://dgjsxb.ces-transaction.com/CN/10.19595/j.cnki.1000-6753.tces.240529 https://dgjsxb.ces-transaction.com/CN/Y0/V/I/2492930

[1] 韩翔宇, 纽春萍, 何海龙, 等. 电磁式断路器状态监测与智能评估技术综述[J]. 电工技术学报, 2023, 38(8): 2191-2210.
Han Xiangyu, Niu Chunping, He Hailong, et al.Review of condition monitoring and intelligent assessment of electromagnetic circuit breaker[J]. Transactions of China Electrotechnical Society, 2023, 38(8): 2191-2210.
[2] 苟军年, 杜愫愫, 刘力. 基于改进掩膜区域卷积神经网络的输电线路绝缘子自爆检测[J]. 电工技术学报, 2023, 38(1): 47-59.
Gou Junnian, Du Susu, Liu Li.Transmission line insulator self-explosion detection based on improved mask region-convolutional neural network[J]. Transactions of China Electrotechnical Society, 2023, 38(1): 47-59.
[3] 杨帆, 王梦珺, 谭天, 等. 基于目标像素宽度识别的电力设备红外成像单目测距改进算法[J]. 电工技术学报, 2023, 38(8): 2244-2254.
Yang Fan, Wang Mengjun, Tan Tian, et al.An improved monocular ranging method for infrared image of power equipment based on the pixel width recognition of objects[J]. Transactions of China Electrotechnical Society, 2023, 38(8): 2244-2254.
[4] 李斌, 屈璐瑶, 朱新山, 等. 基于多尺度特征融合的绝缘子缺陷检测[J]. 电工技术学报, 2023, 38(1): 60-70.
Li Bin, Qu Luyao, Zhu Xinshan, et al.Insulator defect detection based on multi-scale feature fusion[J]. Transactions of China Electrotechnical Society, 2023, 38(1): 60-70.
[5] 普子恒, 张隆, 余欣芸, 等. 500 kV换流变压器无人机巡检路径规划[J]. 电工技术学报, 2023, 38(增刊1): 204-213.
Pu Ziheng, Zhang Long, Yu Xinyun, et al.Route planning of unmanned aerial vehicle patrol inspection for 500 kV converter transformer[J]. Transactions of China Electrotechnical Society, 2023, 38(S1): 204-213.
[6] 李浩伟, 张楚岩, 史梓男, 等. 基于红外图像检测和改进YOLOv4的高压套管故障识别方法[J]. 高压电器, 2023, 59(11): 24-31.
Li Haowei, Zhang Chuyan, Shi Zinan, et al.Fault identification method of high-voltage bushing based on infrared image detection and improved YOLOv4 algorithm[J]. High Voltage Apparatus, 2023, 59(11): 24-31.
[7] Ma Jiayi, Ma Yong, Li Chang.Infrared and visible image fusion methods and applications: a survey[J]. Information Fusion, 2019, 45: 153-178.
[8] 李云红, 罗雪敏, 苏雪平, 等. 基于改进曲率尺度空间算法的电力设备红外与可见光图像配准[J]. 激光与光电子学进展, 2022, 59(12): 1210010.
Li Yunhong, Luo Xuemin, Su Xueping, et al.Registration method for power equipment infrared and visible images based on improved curvature scale space algorithm[J]. Laser & Optoelectronics Progress, 2022, 59(12): 1210010.
[9] 孙浩, 王朋. 一种基于区域划分的改进ORB算法[J]. 北京航空航天大学学报, 2020, 46(9): 1763-1769.
Sun Hao, Wang Peng.An improved ORB algorithm based on region division[J]. Journal of Beijing University of Aeronautics and Astronautics, 2020, 46(9): 1763-1769.
[10] 唐泽恬, 丁召, 曾瑞敏, 等. 基于改进的Harris和二次归一化互相关的量子图像拼接算法[J]. 激光与光电子学进展, 2020, 57(10): 101005.
Tang Zetian, Ding Zhao, Zeng Ruimin, et al.Quantum image stitching algorithm based on improved Harris and quadratic normalized cross correlation[J]. Laser & Optoelectronics Progress, 2020, 57(10): 101005.
[11] Sedaghat A, Mokhtarzade M, Ebadi H.Uniform robust scale-invariant feature matching for optical remote sensing images[J]. IEEE Transactions on Geoscience and Remote Sensing, 2011, 49(11): 4516-4527.
[12] Bay H, Ess A, Tuytelaars T, et al.Speeded-up robust features (SURF)[J]. Computer Vision and Image Understanding, 2008, 110(3): 346-359.
[13] Chen Jian, Tian Jie, Lee N, et al.A partial intensity invariant feature descriptor for multimodal retinal image registration[J]. IEEE Transactions on Bio-Medical Engineering, 2010, 57(7): 1707-1718.
[14] Li Ning, Li Yuxuan, Jiao Jichao. Multimodal remote sensing image registration based on adaptive multi-scale PIIFD[J/OL]. Multimedia Tools and Applications, 2024: 1-13[2024-05-16]. https://doi.org/10.1007/s11042-024-18756-1.
[15] Sun Wanfeng, Gao Haibo, Li Cheng.A two-stage registration strategy for thermal-visible images in substations[J]. Applied Sciences, 2024, 14(3): 1158.
[16] 朱英宏, 李俊山, 汤雨. 基于CSS角点提取的红外与可见光图像匹配算法[J]. 系统工程与电子技术, 2011, 33(11): 2540-2545.
Zhu Yinghong, Li Junshan, Tang Yu.Matching algorithm for IR/visible images based on CSS corner extraction[J]. Systems Engineering and Electronics, 2011, 33(11): 2540-2545.
[17] 姜骞, 刘亚东, 方健, 等. 基于轮廓特征的电力设备红外和可见光图像配准方法[J]. 仪器仪表学报,2020, 41(11): 252-260.
Jiang Qian, Liu Yadong, Fang Jian, et al.Registration method for power equipment infrared and visible images based on contour feature[J]. Chinese Journal of Scientific Instrument, 2020, 41(11): 252-260.
[18] Wang Zhengbing, Feng Xugang, Xu Guili, et al.A robust visible and infrared image matching algorithm for power equipment based on phase congruency and scale-invariant feature[J]. Optics and Lasers in Engineering, 2023, 164: 107517.
[19] Aguilera C A, Sappa A D, Toledo R.LGHD: a feature descriptor for matching across non-linear intensity variations[C]//2015 IEEE International Conference on Image Processing (ICIP), Quebec City, QC, Canada, 2015: 178-181.
[20] Li Jiayuan, Hu Qingwu, Ai Mingyao.RIFT: multi-modal image matching based on radiation-variation insensitive feature transform[J]. IEEE Transactions on Image Processing, 2020, 29: 3296-3310.
[21] DeTone D, Malisiewicz T, Rabinovich A. SuperPoint: self-supervised interest point detection and description[C]//2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), Salt Lake City, UT, USA, 2018: 224-236.
[22] Sarlin P E, DeTone D, Malisiewicz T, et al. SuperGlue: learning feature matching with graph neural networks[C]//2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Seattle, WA, USA, 2020: 4938-4947.
[23] Shi Yan, Cai Junxiong, Shavit Y, et al.ClusterGNN: cluster-based coarse-to-fine graph neural network for efficient feature matching[C]//2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), New Orleans, LA, USA, 2022: 12517-12526.
[24] Zhang Shizheng, Li Baohuan, Zhang Zhifeng, et al.Robust corner finding based on multi-scale K-cosine angle detection[J]. IEEE Access, 2020, 8: 66741-66748.
[25] Zhong Baojiang, Liao Wenhe.Direct curvature scale space: theory and corner detection[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2007, 29(3): 508-512.
[26] Li Dongqing, Zhong Baojiang, Ma Kaikuang.Multi-scale corner detection based on arithmetic mean curvature[C]//2015 IEEE China Summit and International Conference on Signal and Information Processing (ChinaSIP), Chengdu, China, 2015: 433-437.
[27] Rosenfeld A, Johnston E. Angle detection on digital curves[J]. IEEE Transactions on Computers, 1973, C-22(9): 875-878.
[28] Wang Zhengbing, Feng Xugang, Li Dan, et al.A robust multi-source image matching method for power equipment based on improved PIIFD[C]//2023 8th International Conference on Signal and Image Processing (ICSIP), Wuxi, China, 2023: 254-258.
[29] Gao Chenzhong, Li Wei.Multi-scale PIIFD for registration of multi-source remote sensing images[J]. Journal of Beijing Institute of Technology, 2021, 30(2): 113-124.
[30] 谭建豪, 殷旺, 刘力铭, 等. 引入全局上下文特征模块的DenseNet孪生网络目标跟踪[J]. 电子与信息学报, 2021, 43(1): 179-186.
Tan Jianhao, Yin Wang, Liu Liming, et al.DenseNet-Siamese network with global context feature module for object tracking[J]. Journal of Electronics & Information Technology, 2021, 43(1): 179-186.
[31] Deng Haowen, Birdal T, Ilic S.PPFNet: global context aware local features for robust 3D point matching[C]//2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, 2018: 195-205.
[32] 朱惠玲, 牛哲文, 黄克灿, 等. 基于单阶段目标检测算法的变电设备红外图像目标识别及定位[J]. 电力自动化设备, 2021, 41(8): 217-224.
Zhu Huiling, Niu Zhewen, Huang Kecan, et al.Identification and location of infrared image for substation equipment based on single-stage object detection algorithm[J]. Electric Power Automation Equipment, 2021, 41(8): 217-224.