Infrared Thermal Image Enhanced Recognition Method for Substation Equipment Based on Multi-Modal Image Fusion
Yang Fan1, Li Zhimin1, Li Yan2, Tian Jie2, Yi Yong2
1. State Key Laboratory of Power Transmission Equipment Technology Chongqing University Chongqing 400044 China; 2. Electric Power Research Institute Shenzhen Power Supply Co. Ltd Shenzhen 518000 China
Abstract：The automatic recognition of substation equipment in infrared images is a crucial step for defect and fault diagnosis. However, current methods for substation equipment recognition in infrared images faces challenges such as low accuracy and poor efficiency, primarily due to the complex background of the equipment and the limited information content of single-modal images. To address this issue, this paper proposes an infrared thermal image enhanced recognition method for substation equipment based on multi-modal image fusion. First, infrared and visible images of five classes of equipment, including arresters, breakers, current transformers, potential transformers, and insulators, are collected, covering multiple voltage levels such as 110 kV, 220 kV, and 500 kV. Subsequently, an unsupervised paradigm for multi-modal image fusion is established. By optimizing the image fusion architecture, feature fusion is integrated into each processing stage. The nonlinear fitting capability of deep convolutional neural networks is leveraged to enhance the adaptive ability of feature fusion. Furthermore, a multi-dimensional cross-connection mechanism is introduced to improve feature flow and utilization efficiency, thereby achieving effective fusion of infrared and visible images. Additionally, the feature extraction network of YOLOv7 is enhanced. A feature preservation strategy is implemented to minimize the loss of detailed information, such as small targets and low-contrast regions, during feature extraction. Simultaneously, a feature cross-connection mechanism is incorporated to strengthen information transfer between shallow and deep features, improving the utilization efficiency of multi-modal features and consequently increasing the recognition accuracy of substation equipment. The experimental results demonstrate that the unsupervised image fusion model can effectively integrate salient features from both infrared and visible images, exhibiting significant advantages over mainstream methods across four evaluation metrics, including information entropy. Second, the recognition of substation equipment using fused images significantly enhances detection performance, with an increase of 4.08 percentage points in mean average precision, 10.40 percentage points in average recall, 7.65 percentage points in average F1-score, and 5.70 percentage points in overall accuracy. Moreover, compared to the original YOLOv7 network, the enhanced network demonstrates improved performance in substation equipment recognition, achieving a 1.37 percentage points increase in mean average precision, a 0.33 percentage points improvement in F1-score, a 0.61 percentage points enhancement in accuracy, and a 0.28 ms acceleration in average detection speed. Finally, through comparative analysis of recognition results between infrared and fused images under varying environmental conditions, including illumination and weather variations, the robustness of the proposed method is effectively validated. Based on these results, the following conclusions can be drawn: (1) The proposed method effectively fuses infrared and visible images of substation equipment, enriching the feature information of the equipment in the fused images. (2) The proposed method outperforms single-modal recognition approaches for substation equipment in recognition accuracy and confidence levels. (3) The proposed method effectively mitigates the influence of complex environmental factors, such as lighting and temperature variations, on recognition performance, thereby improving the robustness of substation equipment recognition.
杨帆, 李致民, 李艳, 田杰, 怡勇. 基于多模态图像融合的变电设备红外热像增强识别方法[J]. 电工技术学报, 2026, 41(3): 975-986.
Yang Fan, Li Zhimin, Li Yan, Tian Jie, Yi Yong. Infrared Thermal Image Enhanced Recognition Method for Substation Equipment Based on Multi-Modal Image Fusion. Transactions of China Electrotechnical Society, 2026, 41(3): 975-986.
[1] 肖懿, 罗丹, 蒋沁知, 等. 基于温度概率密度的变电站高压设备故障热红外图像识别方法[J]. 高电压技术, 2022, 48(1): 307-318. Xiao Yi, Luo Dan, Jiang Qinzhi, et al.Thermal infrared image recognition method for high voltage equipment failure in substation based on temperature probability density[J]. High Voltage Engineering, 2022, 48(1): 307-318. [2] 杨帆, 王梦珺, 谭天, 等. 基于目标像素宽度识别的电力设备红外成像单目测距改进算法[J]. 电工技术学报, 2023, 38(8): 2244-2254. Yang Fan, Wang Mengjun, Tan Tian, et al.An improved monocular ranging method for infrared image of power equipment based on the pixel width recognition of objects[J]. Transactions of China Electrotechnical Society, 2023, 38(8): 2244-2254. [3] 刘云鹏, 王权, 刘一瑾, 等. 面向电力设备红外图像的混合噪声去除方法研究[J]. 高压电器, 2024, 60(3): 186-192. Liu Yunpeng, Wang Quan, Liu Yijin, et al.Research on mixed noise removal method for infrared image of power equipment[J]. High Voltage Apparatus, 2024, 60(3): 186-192. [4] 裴少通, 张行远, 胡晨龙, 等. 基于ER-YOLO算法的跨环境输电线路缺陷识别方法[J]. 电工技术学报, 2024, 39(9): 2825-2840. Pei Shaotong, Zhang Hangyuan, Hu Chenlong, et al.The defect detection method for cross-environment power transmission line based on ER-YOLO algorithm[J]. Transactions of China Electrotechnical Society, 2024, 39(9): 2825-2840. [5] 赵振兵, 冯烁, 席悦, 等. 大模型时代: 电力视觉技术新起点[J]. 高电压技术, 2024, 50(5): 1813-1825. Zhao Zhenbing, Feng Shuo, Xi Yue, et al.The era of large models: a new starting point for electric power vision technology[J]. High Voltage Engineering, 2024, 50(5): 1813-1825. [6] 刘传洋, 吴一全. 基于红外图像的电力设备识别及发热故障诊断方法研究进展[J]. 中国电机工程学报, 2025, 45(6): 2171-2196. Liu Chuanyang, Wu Yiquan.Research progress of power equipment identification and thermal fault diagnosis based on infrared images[J]. Proceedings of the CSEE, 2025, 45(6): 2171-2196. [7] 叶俊, 邱志斌, 黄玉水, 等. 基于可见光与红外图像融合的变电设备轻量化检测算法[J]. 电气工程学报, 2025, 20(6): 30-41. Ye Jun, Qiu Zhibin, Huang Yushui, et al.Lightweight detection algorithm for substation equipment based on visible and infrared image fusion[J]. Journal of Electrical Engineering, 2025, 20(6): 30-41. [8] 余泊羲, 杨昊, 刘志业, 等. 基于主动红外热波技术的环氧复合材料内部裂纹缺陷检测可行性研究[J]. 绝缘材料, 2025, 58(11): 122-130. Yu Boxi, Yang Hao, Liu Zhiye, et al.Feasibility study on internal crack defect detection in epoxy composites based on active infrared thermal wave technology[J]. Insulating Materials, 2025, 58(11): 122-130. [9] 姜香菊, 王瑞彤, 马彦鸿. 基于轻量级改进RT-DETR边缘部署算法的绝缘子缺陷检测[J]. 电工技术学报, 2025, 40(3): 842-854. Jiang Xiangju, Wang Ruitong, Ma Yanhong.Insulator defect detection based on lightweight improved RT-DETR edge deployment algorithm[J]. Transactions of China Electrotechnical Society, 2025, 40(3): 842-854. [10] 刘传洋, 吴一全, 刘景景. 无人机航拍图像中绝缘子缺陷检测的深度学习方法研究进展[J]. 电工技术学报, 2025, 40(9): 2897-2916. Liu Chuanyang, Wu Yiquan, Liu Jingjing.Research progress of deep learning methods for insulator defect detection in UAV based aerial images[J]. Trans-actions of China Electrotechnical Society, 2025, 40(9): 2897-2916. [11] 仲林林, 刘柯妤. 面向电力巡检图像目标检测的联邦学习激励机制[J]. 电工技术学报, 2024, 39(17): 5434-5449. Zhong Linlin, Liu Keyu.Federated-learning incentive mechanism for object detection in power inspection images[J]. Transactions of China Electrotechnical Society, 2024, 39(17): 5434-5449. [12] 孙瑞筱, 胡玉耀, 蒋兴良, 等. 不同污秽影响下劣化瓷绝缘子发热规律与红外特征分析[J]. 电工技术学报, 2025, 40(11): 3591-3603. Sun Ruixiao, Hu Yuyao, Jiang Xingliang, et al.Analysis of temperature rise and infrared feature of faulty porcelain insulators under different contamina-tion effects[J]. Transactions of China Electrotechnical Society, 2025, 40(11): 3591-3603. [13] 裴少通, 马子儒, 刘云鹏, 等. 500 kV输电线路劣化瓷绝缘子非线性发热特性的红外分析[J]. 高压电器, 2024, 60(1): 33-40. Pei Shaotong, Ma Ziru, Liu Yunpeng, et al.Infrared analysis of nonlinear heating characteristics of deteriorated porcelain insulator of 500 kV transmi-ssion lines[J]. High Voltage Apparatus, 2024, 60(1): 33-40. [14] 冯旭刚, 阮善会, 王正兵, 等. 基于局部和全局特征的电力设备红外和可见光图像匹配方法[J]. 电工技术学报, 2025, 40(7): 2236-2246, 2305. Feng Xugang, Ruan Shanhui, Wang Zhengbing, et al.Infrared and visible image matching method for power equipment based on local and global features[J]. Transactions of China Electrotechnical Society, 2025, 40(7): 2236-2246, 2305. [15] Li Daoliang, Song Zhaoyang, Quan Chaoqun, et al.Recent advances in image fusion technology in agriculture[J]. Computers and Electronics in Agricul-ture, 2021, 191: 106491. [16] Li Hui, Wu Xiaojun, Kittler J.MDLatLRR: a novel decomposition method for infrared and visible image fusion[J]. IEEE Transactions on Image Processing, 2020, 29: 4733-4746. [17] Li Hui, Wu Xiaojun.DenseFuse: a fusion approach to infrared and visible images[J]. IEEE Transactions on Image Processing, 2019, 28(5): 2614-2623. [18] Ma Jiayi, Xu Han, Jiang Junjun, et al.DDcGAN: a dual-discriminator conditional generative adversarial network for multi-resolution image fusion[J]. IEEE Transactions on Image Processing, 2020, 29: 4980-4995. [19] Zhang Yu, Liu Yu, Sun Peng, et al.IFCNN: a general image fusion framework based on convolutional neural network[J]. Information Fusion, 2020, 54: 99-118. [20] Ren Long, Pan Zhibin, Cao Jianzhong, et al.Infrared and visible image fusion based on variational auto-encoder and infrared feature compensation[J]. Infrared Physics & Technology, 2021, 117: 103839. [21] Long Yongzhi, Jia Haitao, Zhong Yida, et al.RXDNFuse: a aggregated residual dense network for infrared and visible image fusion[J]. Information Fusion, 2021, 69: 128-141. [22] 李文璞, 毛颖科, 廖逍, 等. 基于旋转目标检测的变电设备红外图像电压致热型缺陷智能诊断方法[J]. 高电压技术, 2021, 47(9): 3246-3253. Li Wenpu, Mao Yingke, Liao Xiao, et al.Intelligent diagnosis method of infrared image for substation equipment voltage type thermal defects based on rotating target detection[J]. High Voltage Engineering, 2021, 47(9): 3246-3253. [23] 叶剑涛, 杨为, 柯艳国, 等. 基于红外图像实例分割的敞开式开关设备发热缺陷智能诊断方法[J]. 高压电器, 2021, 57(12): 201-208. Ye Jiantao, Yang Wei, Ke Yanguo, et al.Intelligent diagnosis method for thermal defects of air insulated switchgear based on infrared image instance segmenta-tion[J]. High Voltage Apparatus, 2021, 57(12): 201-208. [24] Zhou Fangrong, Ma Yi, Wang Bo, et al.Dual-channel convolutional neural network for power edge image recognition[J]. Journal of Cloud Computing, 2021, 10(1): 18. [25] 谭宇璇, 樊绍胜. 基于图像增强与深度学习的变电设备红外热像识别方法[J]. 中国电机工程学报, 2021, 41(23): 7990-7998. Tan Yuxuan, Fan Shaosheng.Infrared thermal image recognition of substation equipment based on image enhancement and deep learning[J]. Proceedings of the CSEE, 2021, 41(23): 7990-7998. [26] 郑含博, 李金恒, 刘洋, 等. 基于改进YOLOv3的电力设备红外目标检测模型[J]. 电工技术学报, 2021, 36(7): 1389-1398. Zheng Hanbo, Li Jinheng, Liu Yang, et al.Infrared object detection model for power equipment based on improved YOLOv3[J]. Transactions of China Electro-technical Society, 2021, 36(7): 1389-1398. [27] Wang C Y, Bochkovskiy A, Liao H M.YOLOv7: trainable bag-of-freebies sets new state-of-the-art for real-time object detectors[C]//2023 IEEE/CVF Confe-rence on Computer Vision and Pattern Recognition (CVPR), Vancouver, BC, Canada, 2023: 7464-7475. [28] Wang Zhishe, Wu Yuanyuan, Wang Junyao, et al.Res2Fusion: infrared and visible image fusion based on dense Res2net and double nonlocal attention models[J]. IEEE Transactions on Instrumentation Measurement, 2022, 71: TIM.2021. [29] Li Hui, Xu Tianyang, Wu Xiaojun, et al.LRRNet: a novel representation learning guided fusion network for infrared and visible images[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2023, 45(9): 11040-11052. [30] Ma Jiayi, Tang Linfeng, Fan Fan, et al.SwinFusion: cross-domain long-range learning for general image fusion via swin transformer[J]. CAA Journal of Auto-matica Sinica, 2022, 9(7): 1200-1217. [31] Tang Linfeng, Yuan Jiteng, Ma Jiayi.Image fusion in the loop of high-level vision tasks: a semantic-aware real-time infrared and visible image fusion network[J]. Information Fusion, 2022, 82: 28-42.
2026, Vol. 41 
