Magnetic Field Prediction Method Based on Residual U-Net and Self-Attention Transformer Encoder
Jin Liang1,2, Yin Zhenhao1,2, Liu Lu1, Song Juheng1, Liu Yuankai1
1. State Key Laboratory of Reliability and Intelligence of Electrical Equipment Hebei University of Technology Tianjin 300401 China; 2. Key Laboratory of Electromagnetic Field and Electrical Apparatus Reliability of Hebei Province Hebei University of Technology Tianjin 300401 China
Abstract：Accurate simulation of electromagnetic characteristics in electrical equipment relies on the finite element method. However, the increasing complexity of large electrical machines and transformers poses challenges, leading to prolonged simulation time and significant computational resource consumption. At the same time, the finite element method cannot establish a priori model. When design parameters, structures, or operating conditions change, it is necessary to reestablish the model. Considering the powerful feature extraction ability of deep learning, this paper proposes a magnetic field prediction method based on a residual U-Net and a self-attention Transformer encoder. The finite element method is used to obtain the dataset for deep learning training. The deep learning model can be trained once and used for multiple predictions, addressing the limitations of the finite element method and reducing computational time and resource consumption. Firstly, this paper leverages the inherent advantages of the convolutional neural network (CNN) in image processing, particularly the U-shaped CNN, known as U-Net, based on the encoder and decoder structure. This architecture exhibits a stronger ability to capture fine details and learn from limited samples than the traditional CNN. To mitigate network degradation and address convolutional operation limitations, short residual connections and Transformer modules are introduced to the U-Net architecture, creating the ResUnet- Transformer model. The short residual connections accelerate network training, while the self-attention mechanism from the Transformer network facilitates the effective interaction of global features. Secondly, this paper introduces the Targeted Dropout algorithm and adaptive learning rate to suppress overfitting and enhance the accuracy of magnetic field predictions. The Targeted Dropout algorithm incorporates post-pruning strategies into the training process of neural networks, effectively mitigating overfitting and improving the model’s generalization. Additionally, an adaptive learning rate is implemented using the cosine annealing algorithm based on the Adam optimization algorithm, gradually reducing the learning rate as the objective function converges to the optimal value and avoiding oscillations or non-convergence. Finally, the ResUnet-Transformer model is validated through engineering cases involving permanent magnet synchronous motors (PMSM) and amorphous metal transformers (AMT). On the PMSM dataset, training the ResUnet-Transformer model with 250 samples and testing it with 100 samples, the mean square error (MSE) and mean absolute percentage error (MAPE) are used as performance evaluation metrics. Compared to CNN, U-Net, and Linknet models, the ResUnet-Transformer model achieves the highest prediction accuracy, with an MSE of 0.07×10-3 and a MAPE of 1.4%. The prediction efficiency of the 100 test samples using the ResUnet-Transformer model surpasses the finite element method by 66.1%. Maintaining consistency in structural and parameter settings, introducing the Targeted Dropout algorithm and cosine annealing algorithm improves the prediction accuracy by 36.4% and 26.3%, respectively. To evaluate the model's generalization capability, the number of training samples for PMSM and AMT datasets is varied, and the model is tested using 100 samples. Inadequate training samples result in poor magnetic field prediction performance. When the training dataset size increases to 300, the prediction error does not decrease but shows a slight rise. However, with further increases in the training dataset size, the error significantly decreases, and the MAPE for the PMSM and AMT datasets reaches 0.7% and 0.5%, respectively, with just 500 training samples.
金亮, 尹振豪, 刘璐, 宋居恒, 刘元凯. 基于残差U-Net和自注意力Transformer编码器的磁场预测方法[J]. 电工技术学报, 2024, 39(10): 2937-2952.
Jin Liang, Yin Zhenhao, Liu Lu, Song Juheng, Liu Yuankai. Magnetic Field Prediction Method Based on Residual U-Net and Self-Attention Transformer Encoder. Transactions of China Electrotechnical Society, 2024, 39(10): 2937-2952.
[1] 金亮, 王飞, 杨庆新, 等. 永磁同步电机性能分析的典型深度学习模型与训练方法[J]. 电工技术学报, 2018, 33(增刊1): 41-48. Jin Liang, Wang Fei, Yang Qingxin, et al.Typical deep learning model and training method for per- formance analysis of permanent magnet synchronous motor[J]. Transactions of China Electrotechnical Society, 2018, 33(S1): 41-48. [2] 杨帆, 吴涛, 廖瑞金, 等. 数字孪生在电力装备领域中的应用与实现方法[J]. 高电压技术, 2021, 47(5): 1505-1521. Yang Fan, Wu Tao, Liao Ruijin, et al.Application and implementation of digital twins in the field of electric equipment[J]. High Voltage Engineering, 2021, 47(5): 1505-1521. [3] Zheng Weiying, Cheng Zhiguang.An inner-constrained separation technique for 3-D finite-element modeling of grain-oriented silicon steel laminations[J]. IEEE Transactions on Magnetics, 2012, 48(8): 2277-2283. [4] Silver D, Huang A, Maddison C J, et al.Mastering the game of Go with deep neural networks and tree search[J]. Nature, 2016, 529(7587): 484-489. [5] Silver D, Schrittwieser J, Simonyan K, et al.Mastering the game of Go without human knowledge[J]. Nature, 2017, 550(7676): 354-359. [6] 赵洋, 王瀚墨, 康丽, 等. 基于时间卷积网络的短期电力负荷预测[J]. 电工技术学报, 2022, 37(5): 1242-1251. Zhao Yang, Wang Hanmo, Kang Li, et al.Temporal convolution network-based short-term electrical load forecasting[J]. Transactions of China Electrotechnical Society, 2022, 37(5): 1242-1251. [7] 张翼, 朱永利. 结合知识蒸馏和图神经网络的局部放电增量识别方法[J]. 电工技术学报, 2023, 38(5): 1390-1400. Zhang Yi, Zhu Yongli.Partial discharge incremental recognition method combining knowledge distillation and graph neural network[J]. Transactions of China Electrotechnical Society, 2023, 38(5): 1390-1400. [8] LeCun Y, Bengio Y, Hinton G. Deep learning[J]. Nature, 2015, 521(7553): 436-444. [9] 王艳新, 闫静, 王建华, 等. 基于域对抗迁移卷积神经网络的小样本GIS绝缘缺陷智能诊断方法[J]. 电工技术学报, 2022, 37(9): 2150-2160. Wang Yanxin, Yan Jing, Wang Jianhua, et al.Intelligent diagnosis for GIS with small samples using a novel adversarial transfer learning in convolutional neural network[J]. Transactions of China Electro- technical Society, 2022, 37(9): 2150-2160. [10] Krizhevsky A, Sutskever I, Hinton G.Imagenet classification with deep convolutional neural net- works[J]. Communications of the ACM, 2017, 60(6): 84-90. [11] Hinton G, Deng Li, Yu Dong, et al.Deep neural networks for acoustic modeling in speech recognition: the shared views of four research groups[J]. IEEE Signal Processing Magazine, 2012, 29(6): 82-97. [12] Long J, Shelhamer E, Darrell T.Fully convolutional networks for semantic segmentation[J]. IEEE Transa- ctions on Pattern Analysis and Machine Intelligence, 2015, 39(4): 640-651. [13] Ronneberger O, Fischer P, Brox T.U-Net: convolu- tional networks for biomedical image segmentation[C]// 18th International Conference on Medical Image Computing and Computer-Assisted Intervention (MICCAI), 2015: 234-241. [14] He Kaiming, Zhang Xianyu, Ren Shaoqing, et al.Deep residual learning for image recognition[C]// IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2016: 770-778. [15] Zhang Zhengxin, Liu Qingjie, Wang Yunhong.Road extraction by deep residual U-net[J]. IEEE Geoscience and Remote Sensing Letters, 2018, 15(5): 749-753. [16] 徐宏伟, 闫培新, 吴敏, 等. 基于残差双注意力U-Net模型的CT图像囊肿肾脏自动分割[J]. 计算机应用研究, 2020, 37(7): 2237-2240. Xu Hongwei, Yan Peixin, Wu Min, et al.Automated segmentation of cystic kidney in CT images using residual double attention motivated U-Net model[J]. Application Research of Computers, 2020, 37(7): 2237-2240. [17] 李耀仟, 李才子, 刘瑞强, 等. 面向手术器械语义分割的半监督时空Transformer网络[J]. 软件学报, 2022, 33(4): 1501-1515. Li Yaoqian, Li Caizi, Liu Ruiqiang, et al.Semi- supervised spatiotemporal transformer network for semantic segmentation of surgical instruments[J]. Journal of Software, 2022, 33(4): 1501-1515. [18] Vaswani A, Shazeer N, Parmar N, et al.Attention is all you need[C]//Advances in Neural Information Processing Systems, 2017: 5998-6008. [19] Dosovitskiy A, Beyer L, Kolesnikov A, et al.An image is worth 16x16 words: transformers for image recognition at scale[J]. arXiv: 2010.11929, 2020. [20] Chen Jieneng, Lu Yongyi, Yu Qihang, et al.TransUNet: transformers make strong encoders for medical image segmentation[J]. arXiv: 2102.04306, 2021. [21] Saakaar B, Yaser A, Shaowu P, et al.Deep learning for magnetic field estimation[J]. IEEE Transactions on Magnetics, 2019, 55(6): 1-4. [22] Gong Ruohai, Tang Zuqi.Investigation of convolu- tional neural network U-net under small datasets in transformer magneto-thermal coupled analysis[J]. COMPEL: The International Journal for Computation and Mathematics in Electrical and Electronic Engineering, 2020, 39(4): 959-970. [23] Li Y, Wang Y, Qi S, et al.Predicting scattering from complex nano-structures via deep learning[J]. IEEE Access, 2020, 8: 139983-139993. [24] 吴鑫, 张艳丽, 王振, 等. 基于深度学习的电工钢片磁畴磁化过程预测与特征量提取[J]. 电工技术学报, 2023, 38(9): 2289-2298. Wu Xin, Zhang Yanli, Wang Zhen, et al.Prediction of domain magnetization process and feature extraction of electrical steel sheet based on deep learning[J]. Transactions of China Electrotechnical Society, 2023, 38(9): 2289-2298. [25] Xiaowei J, Cheng Peng, Chen Wenli, et al.Prediction model of velocity field around circular cylinder over various Reynolds numbers by fusion convolutional neural networks based on pressure on the cylinder[J]. Physics of Fluids, 2018, 30(4): 1-16. [26] Vinothkumar S, Qinghua J, Chang S, et al.Fast flow field prediction over airfoils using deep learning approach[J]. Physics of Fluids, 2019, 31(5): 1-69. [27] Liu Bo, Tang Jiupeng, Huang Haibo, et al.Deep learning methods for super-resolution reconstruction of turbulent flows[J]. Physics of Fluids, 2020, 32(2): 1-13. [28] Bhatnagar, Saakaar, Afshar, et al. Prediction of aerodynamic flow fields using convolutional neural networks[J]. Computational Mechanics, 2019, 64(2): 525-545. [29] 殷晓航, 王永才, 李德英. 基于U-Net结构改进的医学影像分割技术综述[J]. 软件学报, 2021, 32(2): 519-550. Yin Xiaohang, Wang Yongcai, Li Deying.Summary of medical image segmentation technology based on U-Net structure improvement[J]. Journal of Software, 2021, 32(2): 519-550. [30] Liu Zhenqing, Cao Yiwen, Wang Yize, et al.Com- puter vision-based concrete crack detection using U-net fully convolutional networks[J]. Automation in Construction, 2019, 104: 129-139. [31] Ibtehaz N, Rahman M S.MultiResUNet: rethinking the U-Net architecture for multimodal biomedical image segmentation[J]. Neural Networks, 2020, 121: 74-87. [32] Hochreiter S, Schmidhuber J.Long short-term memory[J]. Neural Computation, 1997, 9(8): 1735-1780. [33] 王琛, 王颖, 郑涛, 等. 基于ResNet-LSTM网络和注意力机制的综合能源系统多元负荷预测[J]. 电工技术学报, 2022, 37(7): 1789-1799. Wang Chen, Wang Ying, Zheng Tao, et al.Multi- energy load forecasting in integrated energy system based on ResNet-LSTM network and attention mechanism[J]. Transactions of China Electrotechnical Society, 2022, 37(7): 1789-1799. [34] 房佳姝, 刘崇茹, 苏晨博, 等. 基于自注意力Trans- former编码器的多阶段电力系统暂态稳定评估方法[J]. 中国电机工程学报, 2022: 1-13. Fang Jiashu, Liu Chongru, Su Chenbo, et al.Multi- stage power system transient stability assessment method based on self-attention transformer encoder[J]. Proceedings of the CSEE, 2022: 1-13. [35] Ayers C W, Hsu J S, Marlino L D, et al.Evaluation of 2004 Toyota prius hybrid electic drive system interim report[R]. Oak Ridge National Laboratory, 2004. [36] Burress T A, Campbell S L, Coomer C, et al.Evaluation of the 2010 Toyota prius hybrid synergy drive system[R]. Oak Ridge National Laboratory, 2011. [37] 余光正, 陆柳, 汤波, 等. 基于云图特征提取的改进混合神经网络超短期光伏功率预测方法[J]. 中国电机工程学报, 2021, 41(20): 6989-7003. Yu Guangzheng, Lu Liu, Tang Bo, et al.Improved hybrid neural network ultra-short term photovoltaic power prediction method based on cloud image feature extraction[J]. Proceedings of the CSEE, 2021, 41(20): 6989-7003. [38] 余印振, 韩哲哲, 许传龙. 基于深度卷积神经网络和支持向量机的NOx浓度预测[J]. 中国电机工程学报, 2022, 42(1): 238-248. Yu Yinzhen, Han Zhezhe, Xu Chuanlong.The concentration prediction of Nox based on deep convolution neural network and support vector machine[J]. Proceedings of the CSEE, 2022, 42(1): 238-248. [39] 朱煜峰, 许永鹏, 陈孝信, 等. 基于卷积神经网络的直流XLPE电缆局部放电模式识别技术[J]. 电工技术学报, 2020, 35(3): 659-668. Zhu Yufeng, Xu Yongpeng, Chen Xiaoxin, et al.Partial discharge pattern recognition technology of DC XLPE cable based on convolution neural net- work[J]. Transactions of China Electrotechnical Society, 2020, 35(3): 659-668.
2024, Vol. 39 
