Abstract：Partial discharge (PD) is the primary hidden danger threatening the insulation safety of high-voltage power equipment. Typically, there is some correlation between discharge type and insulation damage, so that by identifying the PD types, a large number of insulation faults can be predicted or detected in a timely manner. Recently, deep learning (DL) technology has been gradually applied in PD, showing excellent performance in PD pattern recognition. However, its learning process terminates after learning all the current data at once, which means that those PD recognition models cannot be gradually trained on the new PD data collected later. To address it, an incremental learning method combining knowledge distillation and graph neural network (GNN) for PD recognition is proposed in this paper, which can gradually expand the generalization ability of the original recognition model. Firstly, a deep neural network (DNN) is trained as the original model M with the old PD data set. Then, according to the knowledge distillation theory, the prior knowledge from M is transferred to avoid forgetting in the process of incremental training by replaying a small amount of old PD data, and meanwhile, the new PD data can be learned with the prior knowledge assistance, which improves the generalization ability of the M. Finally, to adapt to the uncertainty of the new data size, it adopts the GNN layers to extract the abundant correlation information among various types of PD data, making up for the information shortage of limited samples. In this way, the DL-based PD model learns the continuously increasing PD data efficiently without retraining on all the old PD data and achieves better incremental recognition with different set sizes of the new data. The experimental results show that with sufficient new PD data, the proposed incremental PD recognition method increases the accuracy by roughly 18%. In contrast to the traditional knowledge distillation, the proposed method with GNN increases the recognition accuracy by 2.75% to 9.25% on several new datasets with fewer samples, and reduces the adverse effects of unbalanced categories that are normally caused by the randomness of insulation defects. Moreover, the method has excellent generalization properties and is also effective on the incremental updates of other PD recognition models such as AlexNet, ResNet or MobileNet based models. More significantly, it requires less computational resources than retraining, reducing its GPU and RAM footprint by 67.9% and 72.7%, respectively. The following conclusions can be drawn from the experiments analysis: (1) By introducing the knowledge distillation theory, the DL-based PD recognition model can inherit the recognition ability of original PD model as well as learn the new PD data gradually arriving at the monitoring platform, which is beneficial to improve the generalization ability of PD models. (2) The added GNN builds the graph data by randomly combining multiple types of PD samples, which increases the diversity of training samples. Therefore, it is appropriate to apply GNN to incrementally learn limited samples or imbalanced datasets in categories. (3) Compared to retrain model, this method requires less hardware resources in incremental training, making the deployment and local maintenance of the DL-based PD recognition models possible. (4) Furthermore, the proposed method is a universal incremental method so that it is effective on numerous common PD recognition models based on classical DNNs.
[1] 李泽, 王辉, 钱勇, 等. 基于加速鲁棒特征的含噪局部放电模式识别[J]. 电工技术学报, 2022, 37(3): 775-785. Li Ze, Wang Hui, Qian Yong, et al.Pattern recognition of partial discharge in the presence of noise based on speeded up robust features[J]. Transactions of China Electrotechnical Society, 2022, 37(3): 775-785. [2] 邓冉, 朱永利, 刘雪纯, 等. 基于变量预测-谷本相似度方法的局部放电中未知类型信号识别[J]. 电工技术学报, 2020, 35(14): 3105-3115. Deng Ran, Zhu Yongli, Liu Xuechun, et al.Pattern recognition of unknown types in partial discharge signals based on variable predictive model and tanimoto[J]. Transactions of China Electrotechnical Society, 2020, 35(14): 3105-3115. [3] 宋思蒙, 钱勇, 王辉, 等. 基于方向梯度直方图属性空间的局部放电模式识别改进算法[J]. 电工技术学报, 2021, 36(10): 2153-2160. Song Simeng, Qian Yong, Wang Hui, et al.Improved algorithm for partial discharge pattern recognition based on histogram of oriented gradient attribute space[J]. Transactions of China Electrotechnical Society, 2021, 36(10): 2153-2160. [4] Lu Shibo, Chai Hua, Sahoo A, et al.Condition monitoring based on partial discharge diagnostics using machine learning methods: a comprehensive state-of-the-art review[J]. IEEE Transactions on Dielectrics and Electrical Insulation, 2020, 27(6): 1861-1888. [5] Duan Lian, Hu Jun, Zhao Gen, et al.Identification of partial discharge defects based on deep learning method[J]. IEEE Transactions on Power Delivery, 2019, 34(4): 1557-1568. [6] 张重远, 岳浩天, 王博闻, 等. 基于相似矩阵盲源分离与卷积神经网络的局部放电超声信号深度学习模式识别方法[J]. 电网技术, 2019, 43(6): 1900-1907. Zhang Zhongyuan, Yue Haotian, Wang Bowen, et al.Pattern recognition of partial discharge ultrasonic signal based on similar matrix BSS and deep learning CNN[J]. Power System Technology, 2019, 43(6): 1900-1907. [7] Song Hui, Dai Jiejie, Sheng Gehao, et al.GIS partial discharge pattern recognition via deep convolutional neural network under complex data source[J]. IEEE Transactions on Dielectrics and Electrical Insulation, 2018, 25(2): 678-685. [8] 朱煜峰, 许永鹏, 陈孝信, 等. 基于卷积神经网络的直流XLPE电缆局部放电模式识别技术[J]. 电工技术学报, 2020, 35(3): 659-668. Zhu Yufeng, Xu Yongpeng, Chen Xiaoxin, et al.Pattern recognition of partial discharges in DC XLPE cables based on convolutional neural network[J]. Transactions of China Electrotechnical Society, 2020, 35(3): 659-668. [9] Gao Angran, Zhu Yongli, Cai Weihao, et al.Pattern recognition of partial discharge based on VMD-CWD spectrum and optimized CNN with cross-layer feature fusion[J]. IEEE Access, 2020, 8: 151296-151306. [10] Wang Yanxin, Yan Jing, Sun Qifeng, et al.A MobileNets convolutional neural network for GIS partial discharge pattern recognition in the ubiquitous power internet of things context: optimization, comparison, and application[J]. IEEE Access, 2019, 7: 150226-150236. [11] 汤奕, 崔晗, 党杰. 基于继承思想的时变性电力系统暂态稳定预测[J]. 中国电机工程学报, 2021, 41(15): 5107-5119. Tang Yi, Cui Han, Dang Jie.Transient stability prediction of time-varying power systems based on inheritance[J]. Proceedings of the CSEE, 2021, 41(15): 5107-5119. [12] 范兴明, 王超, 张鑫, 等. 基于增量学习相关向量机的锂离子电池SOC预测方法[J]. 电工技术学报, 2019, 34(13): 2700-2708. Fan Xingming, Wang Chao, Zhang Xin, et al.A prediction method of Li-ion batteries SOC based on incremental learning relevance vector machine[J]. Transactions of China Electrotechnical Society, 2019, 34(13): 2700-2708. [13] Li Zhizhong, Hoiem D.Learning without forgetting[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2018, 40(12): 2935-2947. [14] Rebuffi S A, Kolesnikov A, Sperl G, et al.iCaRL: incremental classifier and representation learning[C]//2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Honolulu, HI, USA, 2017: 5533-5542. [15] Castro F M, Marín-Jiménez M J, Guil N, et al. End-to-end incremental learning[C]//Computer Vision - ECCV 2018, Munich, Germany, 2018: 241-257. [16] Cheraghian A, Rahman S, Fang Pengfei, et al.Semantic-aware knowledge distillation for few-shot class-incremental learning[C]//2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Nashville, TN, USA, 2021: 2534-2543. [17] Garcia V, Bruna J.Few-shot learning with graph neural networks[C]//6th International Conference on Learning Representations, Vancouver, Canada, 2018: 1-13. [18] Kim J, Kim T, Kim S, et al.Edge-labeling graph neural network for few-shot learning[C]//2019 IEEE/ CVF Conference on Computer Vision and Pattern Recognition (CVPR), Long Beach, CA, USA, 2020: 11-20. [19] Parisi G I, Kemker R, Part J L, et al.Continual lifelong learning with neural networks: a review[J]. Neural Networks, 2019, 113: 54-71. [20] 赵振兵, 金超熊, 戚银城, 等. 基于动态监督知识蒸馏的输电线路螺栓缺陷图像分类[J]. 高电压技术, 2021, 47(2): 406-414. Zhao Zhenbing, Jin Chaoxiong, Qi Yincheng, et al.Image classification of transmission line bolt defects based on dynamic supervision knowledge distillation[J]. High Voltage Engineering, 2021, 47(2): 406-414. [21] 刘广一, 戴仁昶, 路轶, 等. 基于图计算的能量管理系统实时网络分析应用研发[J]. 电工技术学报, 2020, 35(11): 2339-2348. Liu Guangyi, Dai Renchang, Lu Yi, et al.Graph computing based power network analysis applications[J]. Transactions of China Electrotechnical Society, 2020, 35(11): 2339-2348. [22] Kearnes S, McCloskey K, Berndl M, et al. Molecular graph convolutions: moving beyond fingerprints[J]. Journal of Computer-Aided Molecular Design, 2016, 30(8): 595-608. [23] Zhou Xiang, Shen Fumin, Liu Li, et al.Graph convolutional network hashing[J]. IEEE Transactions on Cybernetics, 2020, 50(4): 1460-1472.
2023, Vol. 38 
