基于残差U-Net和自注意力Transformer编码器的磁场预测方法

doi:10.19595/j.cnki.1000-6753.tces.230265

Abstract
Figure/Table
References (0)
Related Citation (15)

Download: PDF (1335 KB) HTML (1 KB)
Export: BibTeX | EndNote (RIS)

Abstract

Fine simulation of electromagnetic characteristics analysis of electrical equipment relies on the finite element method. Due to the increasing complexity of large electrical machines and transformers, it may take dozens or even hundreds of hours to simulate structurally complex large electrical machines and transformers for improving computational accuracy, resulting in significant consumption of computational resources. At the same time, the finite element method cannot establish a priori model, when design parameters, structures, and operating conditions undergo slight changes, it is necessary to reestablish the model for calculation, which makes it difficult to meet the requirements of practical engineering and digital twins in terms of speed and adaptability. Considering the powerful feature extraction ability of deep learning, it can extract useful information from massive data, this paper proposes a magnetic field prediction method based on residual U-Net and self-attention Transformer encoder. The finite element method is used to obtain the dataset required for deep learning training. The successful deep learning model which can be trained once and used for multiple predictions, is fully utilized for magnetic field prediction problems, to solve the problem of the non-reusability of the finite element method and reduce the computational time and resource consumption.
Firstly, this paper leveraged the inherent advantages of Convolutional Neural Network (CNN) in image processing, particularly the U-shaped CNN known as U-Net based on encoder and decoder structure, which exhibits stronger capability in capturing fine details and learning from limited samples compared to traditional CNN. To mitigate network degradation and address the limitations of convolutional operations, the paper introduced short residual connections and Transformer modules to the U-Net architecture, creating the ResUnet-Transformer model. The short residual connections accelerate network training, while the self-attention mechanism from the Transformer network enables effective interaction of global features. Secondly, this paper introduced the Targeted Dropout algorithm and adaptive learning rate to suppress overfitting and enhance magnetic field prediction accuracy. The Targeted Dropout algorithm incorporates post-pruning strategies into the training process of neural networks, effectively mitigating overfitting and improving model generalization. Additionally, an adaptive learning rate is implemented using the cosine annealing algorithm based on the Adam optimization algorithm. This adaptive adjustment gradually reduces the learning rate as the objective function converges to the vicinity of the optimal value, avoiding oscillations or non-convergence. Finally, the ResUnet-Transformer model was validated through engineering cases involving Permanent Magnet Synchronous Motor (PMSM) and Amorphous Metal Transformer (AMT).
The experimental results show that on the PMSM dataset, training the ResUnet-Transformer model with 250 samples and testing it with 100 samples, the mean square error (MSE) and mean absolute percentage error (MAPE) are used as performance evaluation metrics. Compared to CNN, U-Net, and Linknet models, the ResUnet-Transformer model achieves the highest prediction accuracy, with an MSE of 0.07×10^-3 and MAPE of 1.4%. The prediction efficiency of the 100 test samples using the ResUnet-Transformer model is 66.1% higher than that of the finite element method. By keeping the other structural and parameter settings, as well as data preprocessing, consistent with the ResUnet-Transformer model, the introduction of the Targeted Dropout algorithm and cosine annealing algorithm improves the prediction accuracy by 36.4% and 26.3%, respectively. To assess the model's generalization ability, the number of training samples for PMSM and AMT datasets is varied, and the model is tested using 100 samples. In the validation on the test set, when the number of training samples is small, the magnetic field prediction performance is poor and does not meet the accuracy requirements. When the training dataset size increases to 300, the prediction error does not decrease but slightly rises. However, with further increases in the training dataset size, the error significantly decreases, and the MAPE on the PMSM and AMT datasets can reach 0.7% and 0.5%, respectively, with just 500 training samples.

Key words： Finite element method electromagnetic field deep learning u-net transformer

Received: 08 March 2023

PACS:

TM153

	Service

	E-mail this article
	Add to my bookshelf
	Add to citation manager
	E-mail Alert
	RSS
	Articles by authors
	Jin Liang
	Yin Zhenhao
	Liu Lu
	Song Jvheng
	Liu Yuankai

Cite this article:

Jin Liang,Yin Zhenhao,Liu Lu等. Magnetic Field Prediction Method Based on Residual U-Net and Self-Attention Transformer Encoder[J]. Transactions of China Electrotechnical Society, 0, (): 9021-21.

URL:

https://dgjsxb.ces-transaction.com/EN/10.19595/j.cnki.1000-6753.tces.230265 OR https://dgjsxb.ces-transaction.com/EN/Y0/V/I/9021

[1] 金亮, 王飞, 杨庆新, 等. 永磁同步电机性能分析的典型深度学习模型与训练方法[J]. 电工技术学报, 2018, 33(S1): 41-48.
Jin Liang, Wang Fei, Yang Qingxin, et al.Typical deep learning model and training method for performance analysis of permanent magnet synchronous motor[J]. Transactions of China Electrotechnical Society, 2018, 33(S1): 41-48.
[2] 杨帆, 吴涛, 廖瑞金, 等. 数字孪生在电力装备领域中的应用与实现方法[J]. 高电压技术, 2021, 47(05): 1505-1521.
Yang Fan, Wu Tao, Liao Ruijin, et al.Application and implementation of digital twins in the field of electric equipment[J]. High Voltage Engineering, 2021, 47(05): 1505-1521.
[3] Zheng Weiying, Cheng Zhiguang.An inner-constrained separation technique for 3-D finite-element modeling of grain-oriented silicon steel laminations[J]. IEEE Transactions on Magnetics, 2012,48(8):2277-2283.
[4] Silver D, Huang A, Maddison C J, et al.Mastering the game of Go with deep neural networks and tree search[J]. Nature, 2016, 529(7587): 484-489.
[5] Silver D, Schrittwieser J, Simonyan K, et al.Mastering the game of Go without human knowledge[J]. Nature, 2017, 550(7676): 354-359.
[6] 卢锦玲, 郭鲁豫. 基于改进深度残差收缩网络的电力系统暂态稳定评估[J]. 电工技术学报, 2021, 36(11): 2233-2244.
Lu Jinling, Guo Luyu.Power system transient stability assessment based on improved deep residual shrinkage network[J]. Transactions of China Electrotechnical Society, 2021, 36(11): 2233-2244.
[7] 张翼, 朱永利. 结合知识蒸馏和图神经网络的局部放电增量识别方法[J]. 电工技术学报, 2023, 38(5): 1390-1400.
Zhang Yi, Zhu Yongli.Partial discharge incremental recognition method combining knowledge distillation and graph neural network[J]. Transactions of China Electrotechnical Society, 2023, 38(5): 1390-1400.
[8] LeCun Y, Bengio Y, Hinton G. Deep learning[J]. Nature, 2015, 521(7553): 436-444.
[9] 徐建军, 黄立达, 闫丽梅, 等. 基于层次多任务深度学习的绝缘子自爆缺陷检测[J]. 电工技术学报, 2021, 36(7): 1407-1415.
Xu Jianjun, Huang Lida, Yan Limei, et al.Detection of insulator self explosion defects based on hierarchical multi task deep learning[J]. Transactions of China Electrotechnical Society, 2021, 36(7): 1407-1415.
[10] Krizhevsky A, Sutskever I, Hinton G.Imagenet classification with deep convolutional neural networks[J]. Communications of the ACM, 2017,60(6):84-90.
[11] Hinton G, Deng Li, Yu Dong, et al.Deep neural networks for acoustic modeling in speech recognition: the shared views of four research groups[J]. IEEE Signal Processing Magazine, 2012,29(6):82-97.
[12] Long J, Shelhamer E, Darrell T.Fully Convolutional Networks for Semantic Segmentation[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2015,39(4):640-651.
[13] Ronneberger O, Fischer P, Brox T.U-Net: convolutional networks for biomedical image segmentation[C]// 18th International Conference on Medical Image Computing and Computer-Assisted Intervention (MICCAI), 2015.
[14] He Kaiming, Zhang Xianyu, Ren Shaoqing, et al.Deep residual learning for image recognition[C]// IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2016.
[15] Zhang Zhengxin, Liu Qingjie, Wang Yunhong.Road extraction by deep residual U-net[J]. IEEE Geoscience and Remote Sensing Letters, 2018,15(5):749-753.
[16] 徐宏伟, 闫培新, 吴敏, 等. 基于残差双注意力U-Net模型的CT图像囊肿肾脏自动分割[J]. 计算机应用研究, 2020, 37(7): 2237-2240.
Xu Hongwei, Yan Peixin, Wu Min, et al.Automated segmentation of cystic kidney in CT images using residual double attention motivated U-Net model.[J]. Application Research of Computers, 2020, 37(7): 2237-2240.
[17] 李耀仟, 李才子, 刘瑞强, 等. 面向手术器械语义分割的半监督时空Transformer网络[J]. 软件学报, 2022, 33(4): 1501-1515.
Li Yaoqian, Li Caizi, Liu Ruiqiang, et al.Semi-supervised spatiotemporal transformer network for semantic segmentation of surgical instruments[J]. Journal of Software, 2022, 33(4): 1501-1515.
[18] Vaswani A, Shazeer N, Parmar N, et al. Attention Is All You Need[J]. arXiv:1706.03762v5, 2017.
[19] Dosovitskiy A, Beyer L, Kolesnikov A, et al.An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale[J]. arXiv:2010.11929, 2020.
[20] Chen Jieneng, Lu Yongyi, Yu Qihang, et al.TransUNet: Transformers Make Strong Encoders for Medical Image Segmentation[J]. arXiv:2102.04306, 2021.
[21] Saakaar B, Yaser A, Shaowu P, et al.Deep learning for magnetic field estimation[J]. IEEE Transactions on Magnetics, 2019,55(6):1-4.
[22] Gong Ruohai, Tang Zuqi.Investigation of convolutional neural network U-net under small datasets in transformer magneto-thermal coupled analysis[J]. COMPEL: The international journal for computation and mathematics in electrical and electronic engineering, 2020,39(4):959-970.
[23] Li Y, Wang Y, Qi S, et al.Predicting scattering from complex nano-structures via deep learning[J]. IEEE Access, 2020,8:139983-139993.
[24] 吴鑫, 张艳丽, 王振, 等. 基于深度学习的电工钢片磁畴磁化过程预测与特征量提取[J]. 电工技术学报, 2023, 38(9): 2289-2298.
Wu Xin, Zhang Yanli, Wang Zhen, et al.Prediction of domain magnetization process and feature extraction of Electrical steel sheet based on deep learning[J]. Transactions of China Electrotechnical Society, 2023, 38(9): 2289-2298.
[25] Xiaowei J, Cheng Peng, Chen Wenli, et al.Prediction model of velocity field around circular cylinder over various Reynolds numbers by fusion convolutional neural networks based on pressure on the cylinder[J]. Physics of Fluids, 2018,4(30):1-16.
[26] Vinothkumar S, Qinghua J, Chang S, et al.Fast flow field prediction over airfoils using deep learning approach[J]. Physics of Fluids, 2019,31(5):1-69.
[27] Liu Bo, Tang Jiupeng, Huang Haibo, et al.Deep learning methods for super-resolution reconstruction of turbulent flows[J]. Physics of Fluids, 2020, 32(2): 1-13.
[28] Bhatnagar, Saakaar, Afshar, et al. Prediction of aerodynamic flow fields using convolutional neural networks[J]. Computational Mechanics, 2019, 64(2): 525-545.
[29] 殷晓航, 王永才, 李德英. 基于U-Net结构改进的医学影像分割技术综述[J]. 软件学报, 2021, 32(2): 519-550.
Yin Xiaohang, Wang Yongcai, Li Deying.Summary of medical image segmentation technology based on U-Net structure improvement[J]. Journal of Software, 2021, 32(2): 519-550.
[30] Liu Zhenqing, Cao Yiwen, Wang Yize, et al.Computer vision-based concrete crack detection using U-net fully convolutional networks[J]. Automation in Construction, 2019, 104: 129-139.
[31] Ibtehaz N, Rahman M S.MultiResUNet : Rethinking the U-Net architecture for multimodal biomedical image segmentation[J]. Neural Networks, 2020,121:74-87.
[32] Hochreiter S, Schmidhuber J.Long Short-Term Memory[J]. Neural Computation, 1997,9(8):1735-1780.
[33] Li Chen, Tan Yusong, Chen Wei, et al.ANU-Net: attention-based nested U-Net to exploit full resolution features for medical image segmentation[J]. Computers & Graphics, 2020, 90: 11-20.
[34] 房佳姝, 刘崇茹, 苏晨博, 等. 基于自注意力Transformer编码器的多阶段电力系统暂态稳定评估方法[J]. 中国电机工程学报, 2022: 1-13.
Fang Jiashu, Liu Chongru, Su Chenbo, et al.Multi-stage power system transient stability assessment method based on self-attention transformer encoder[J]. Proceedings of the CSEE, 2022: 1-13.
[35] Ayers C W, Hsu J S, Marlino L D, et al.Evaluation of 2004 Toyota Prius Hybrid Electic Drive System Interim Report[R]. Oak Ridge National Laboratory, 2004.
[36] Burress T A, Campbell S L, Coomer C, et al.Evaluation of the 2010 Toyota Prius Hybrid Synergy Drive System[R]. Oak Ridge National Laboratory, 2011.
[37] 余光正, 陆柳, 汤波, 等. 基于云图特征提取的改进混合神经网络超短期光伏功率预测方法[J]. 中国电机工程学报, 2021, 41(20): 6989-7003.
Yu Guangzheng, Lu Liu, Tang Bo, et al.Improved hybrid neural network ultra-short term photovoltaic power prediction method based on cloud image feature extraction[J]. Proceedings of the CSEE, 2021, 41(20): 6989-7003.
[38] 余印振, 韩哲哲, 许传龙. 基于深度卷积神经网络和支持向量机的NO_x浓度预测[J]. 中国电机工程学报, 2022, 42(01): 238-248.
Yu Yinzhen, Han Zhezhe, Xu Chuanlong.The concentration prediction of No_x based on deep convolution neural network and support vector machine[J]. Proceedings of the CSEE, 2022, 42(01): 238-248.
[39] 朱煜峰, 许永鹏, 陈孝信, 等. 基于卷积神经网络的直流XLPE电缆局部放电模式识别技术[J]. 电工技术学报, 2020, 35(3): 659-668.
Zhu Yufeng, Xu Yongpeng, Chen Xiaoxin, et al.Partial discharge pattern recognition technology of DC XLPE cable based on convolution neural network[J]. Transactions of China Electrotechnical Society, 2020, 35(3): 659-668.