Extensible Parallel Computing Method with Hundreds of Millions of Freedoms for Large-Scale Engineering Electromagnetic Field
Jin Liang1,2, Li Yuzeng1,2, Yang Qingxin1,2, Zhang Chuang1,2, Yan Shuai3
1. State Key Laboratory of Reliability and Intelligence of Electrical Equipment Hebei University of Technology Tianjin 300130 China; 2. Key Hebei Key Laboratory of Electromagnetic Field and Electrical Apparatus Reliability Hebei University of Technology Tianjin 300130 China; 3. Institute of Electrical Engineering Chinese Academy of Sciences Beijing 100081 China
Abstract：Accurate and rapid electromagnetic field calculation is the basis of fine simulation and optimization design of electrical equipment. In this paper, the scalable parallel computing research is carried out on the elastic cluster of high-speed interconnected high-performance cloud platform. OpenMpi is used as the message passing library, and the dual-primal finite element tearing and interconnecting (FETI-DP) method is selected as the domain decomposition algorithm. The parallel computing of eddy current magnetic vector potential A with constant conductivity is achieved by improving the master-slave/peer parallel program framework, which can reduce the complexity of programming and improve the efficiency of parallel computing. The program is written in C language and the scalable parallel computing method is verified by the benchmark problem of international TEAM Problem 7. In this paper, the master-slave/peer-to-peer parallel program framework and the dual-primal finite element tearing and interconnecting FETI-DP method are introduced into the field of electromagnetic computing, which improves the efficiency and scalability of parallel computing, and provides a new practical and theoretical method for large-scale engineering electromagnetic field calculation.
金亮, 李育增, 杨庆新, 张闯, 闫帅. 大规模工程电磁场的亿自由度可扩展并行计算方法[J]. 电工技术学报, 2022, 37(3): 589-598.
Jin Liang, Li Yuzeng, Yang Qingxin, Zhang Chuang, Yan Shuai. Extensible Parallel Computing Method with Hundreds of Millions of Freedoms for Large-Scale Engineering Electromagnetic Field. Transactions of China Electrotechnical Society, 2022, 37(3): 589-598.
[1] Cheng Zhiguang, Norio Takahashi, Yang Sumei, et al.Loss spectrum and electromagnetic behavior of Problem 21 family[J]. IEEE Transactions on Magnetics, 2006, 42(4): 1467-1470. [2] 谢德馨, 程志光, 杨仕友, 等. 对当前计算电磁学发展的观察与思考——参加COMPUMAG2011会议有感[J]. 电工技术学报, 2013, 28(1): 136-141. Xie Dexin, Cheng Zhiguang, Yang Shiyou, et al.Observation and consideration on the current development of computational electromagnetics—in perspective of COMPUMAG2011[J]. Transactions of China Electrotechnical Society, 2013, 28(1): 136-141. [3] Hihat Nabil, Napieralska-Juszczak Ewa, Lecointe Jean-Philippe, et al.Equivalent permeability of step-lap joints of transformer cores: computational and experimental considerations[J]. IEEE Transactions on Magnetics, 2011, 47(1): 244-251. [4] 李立毅, 孙芝茵, 潘东华, 等. 近零磁环境装置现状综述[J]. 电工技术学报, 2015, 30(15): 136-140. Li Liyi, Sun Zhiyin, Pan Donghua, et al.Status reviews for nearly zero magnetic field environment facility[J]. Transactions of China Electrotechnical Society, 2015, 30(15): 136-140. [5] 程志光, 刘涛, 范亚娜, 等. 基于TEAM P21三维杂散场问题建模仿真与验证[J]. 电工技术学报, 2014, 29(9): 194-203. Cheng Zhiguang, Liu Tao, Fan Yana, et al.TEAM P21-based validation of 3-D stray-field modeling and simulation[J]. Transactions of China Electrotechnical Society, 2014, 29(9): 194-203. [6] 刘广一, 戴仁昶, 路轶, 等. 基于图计算的能量管理系统实时网络分析应用研发[J]. 电工技术学报, 2020, 35(11): 2339-2348. Liu Guangyi, Dai Renchang, Lu Yi, et al.Graph computing based power network analysis applications[J]. Transactions of China Electrotechnical Society, 2020, 35(11): 2339-2348. [7] 李雪, 张琳玮, 姜涛, 等. 基于CPU-GPU异构的电力系统静态电压稳定域边界并行计算方法[J]. 电工技术学报, 2021, 36(19): 4070-4084. Li Xue, Zhang Linwei, Jiang Tao, et al.CPU-GPU heterogeneous computing for static voltage stability region boundary in bulk power systems[J]. Transactions of China Electrotechnical Society, 2021, 36(19): 4070-4084. [8] 刘涛, 习金玉, 宋战锋, 等. 基于多核并行计算的永磁同步电机有限集模型预测控制策略[J]. 电工技术学报, 2021, 36(1): 107-119. Liu Tao, Xi Jinyu, Song Zhanfeng, et al.Finite control set model predictive control of permanent magnet synchronous motor based on multi-core parallel computing[J]. Transactions of China Electrotechnical Society, 2021, 36(1): 107-119. [9] 管建和. 电磁场有限元法解释分布式并行计算的研究[D]. 北京: 中国地质大学(北京), 2006. [10] Li Yujia, Jin Jianming.A new dual-primal domain decomposition approach for finite element simulation of 3-D large-scale electromagnetic problems[J]. IEEE Transactions on Antennas and Propagation, 2007, 55(10): 2803-2810. [11] Zhen Peng, Lee JinFa. Non-conformal domain decomposition method with mixed true second order transmission condition for solving large finite antenna arrays[J]. IEEE Transactions on Antennas and Propagation, 2011, 59(5): 1638-1651. [12] Xue Mingfeng, Jin Jianming.A hybrid conformal/ nonconformal domain decomposition method for multi-region electromagnetic modeling[J]. IEEE Transactions on Antennas and Propagation, 2014, 62(4): 2009-2021. [13] Xue Mingfeng, Jin Jianming.Nonconformal FETI-DP methods for large-scale electromagnetic simulation[J]. IEEE Transactions on Antennas and Propagation, 2012, 60(9): 4291-4305. [14] Mario A Echeverri Bautista, Francesca Vipiana, Matteo Alessandro Francavilla, et al. A nonconformal domain decomposition scheme for the analysis of multiscale structures[J]. IEEE Transactions on Antennas and Propagation, 2015, 63(8): 3548-3560. [15] 唐任远, 吴东阳, 谢德馨. 单元级别并行有限元法求解工程涡流场的关键问题研究[J]. 电工技术学报, 2014, 29(5): 1-8. Tang Renyuan, Wu Dongyang, Xie Dexin.Research on the key problem of element by element parallel FEM applied to engineering eddy current analysis[J]. Transactions of China Electrotechnical Society, 2014, 29(5): 1-8. [16] Zheng Weiying, Cheng Zhiguang.An inner-constrained separation technique for 3D finite element modeling of GO silicon steel laminations[J]. IEEE Transactions on Magnetics, 2012, 48(8): 2277-2283. [17] Wang Yao, Jin Jianming, Philip TKrein.Application of the LU recombination method to the FETI-DP method for solving low-frequency multiscale electromagnetic problems[J]. IEEE Transactions on Magnetics, 2013, 49(10): 5346-5355. [18] Van Emden Henson, Ulrike Meier Yang.BoomerAMG: a parallel algebraic multigrid solver and preconditioner[J]. Applied Numerical Mathematics, 2002, 41(1): 155-177. [19] Ralf Hiptmair, Xu Jinchao.Auxiliary space preconditioning for edge elements[J]. IEEE Transactions on Magnetics, 2008, 44(6): 938-941. [20] Zhang Linbo.A parallel algorithm for adaptive local refinement of tetrahedral meshes using bisection[J]. Numerical Mathematics a Journal of Chinese Universities English Series, 2009, 2(1): 65-89. [21] George Karypis, Vipin Kumar.Multilevel k-way partitioning scheme for irregular graphs[J]. Journal of Parallel and Distributed Computing, 1998, 48(1): 96-129. [22] Charbel Farhat, Michel Lesoinne, Patrick LeTallec, et al. FETI-DP: a dual-primal unified FETI method—Part I: a faster alternative to the two-level FETI method[J]. International Journal for Numerical Methods in Engineering, 2001, 50(7): 1523-1544. [23] Xue Mingfeng, Jin Jianming.A preconditioned dual-primal finite element tearing and interconnecting method for solving three-dimensional time-harmonic Maxwell's equations[J]. Journal of Computational Physics, 2014, 274: 920-935. [24] James Ahrens, Kristi Brislawn, Ken Martin, et al.Large-scale data visualization using parallel data streaming[J]. IEEE Computer Graphics and Applications, 2001, 21: 34-41. [25] 史燕琨, 熊华强. 基于伪并行遗传算法的配电网电容器优化配置[J]. 电力系统保护与控制, 2009, 37(20): 57-60. Shi Yankun, Xiong Huaqiang.Optimal capacitor arrangement based on pseudo parallel genetic algorithm[J]. Power System Protection and Control, 2009, 37(20): 57-60. [26] 黄靖, 张晓锋, 叶志浩. 大型船舶电力系统分布式并行潮流计算方法[J]. 电力系统保护与控制, 2011, 39(7): 50-55. Huang Jing, Zhang Xiaofeng, Ye Zhihao.A distributed parallel load flow calculation method for large ship power system[J]. Power System Protection and Control, 2011, 39(7): 50-55. [27] 简金宝, 杨林峰, 全然. 基于改进多中心校正解耦内点法的动态最优潮流并行算法[J]. 电工技术学报, 2012, 27(6): 232-241. Jian Jinbao, Yang Linfeng, Quan Ran.Parallel algorithm of dynamic optimal power flow based on improved multiple centrality corrections decoupling interior point method[J]. Transactions of China Electrotechnical Society, 2012, 27(6): 232-241. [28] David MFernández, Maryam Mehri Dehnavi, Warren JGross, et al.Alternate parallel processing approach for FEM[J]. IEEE Transactions on Magnetics, 2012, 48(2): 399-402. [29] Gabriel Mateescu, Wolfgang Gentzsch, Calvin JRibbens.Hybrid computing—where HPC meets grid and cloud computing[J]. Future Generation Computer Systems, 2011, 27(5): 440-453. [30] Ismail Ari, Nitel Muhtaroglu. Design and implementation of a cloud computing service for finite element analysis[J]. Advances in Engineering Software, 2013, 60-61: 122-135. [31] 符伟. 云计算环境下的线性有限元方法研究[D]. 武汉: 华中科技大学, 2013. [32] Zhang Y, Gallipoli D, Augarde C E.Simulation- based calibration of geotechnical parameters using parallel hybrid moving boundary particle swarm optimization[J]. Computers and Geotechnics, 2009, 36(4): 604-615. [33] 张友良, 冯夏庭, 茹忠亮. 基于区域分解算法的岩土大规模高性能并行有限元系统研究[J]. 岩石力学与工程学报, 2004, 23(21): 3636-3641. Zhang Youliang,Feng Xiating, Ru Zhongliang.Large-scale high performance parallel finite element system based on domain decompositioin method in geomechanics[J]. Chinese Journal of Rock Mechanics and Engineering , 2004, 23(21): 3636-3641. [34] Zhang Youliang, Domenico Gallipoli.Development of an object-oriented parallel finite element code for unsaturated soils[M]. Berlin Heidelberg: Springer, 2007. [35] 金亮, 汪东梅, 邱运涛, 等. 基于弹性云计算集群的国际TEAM Problem 7基准问题计算方法[J]. 电工技术学报, 2017, 32(8): 144-150. Jin Liang, Wang Dongmei, Qiu Yuntao, et al.A calculation method for the international benchmark Problem of TEAM 7 based on elastic cluster in cloud computing[J]. Transactions of China Electrotechnical Society, 2017, 32(8): 144-150. [36] 金亮, 邱运涛, 杨庆新, 等. 基于云计算的电磁问题并行计算方法[J]. 电工技术学报, 2016, 31(22): 5-11. Jin Liang, Qiu Yuntao,Yang Qingxin, et al.A parallel computing method to electromagnetic problems based on cloud computing[J]. Transactions of China Electrotechnical Society, 2016, 31(22): 5-11. [37] 颜威利, 杨庆新, 汪友华, 等. 电气工程电磁场数值分析[M]. 北京:机械工业出版社, 2006. [38] 张友良, 谭飞, 张礼仁, 等. 岩土工程亿级单元有限元模型可扩展并行计算[J]. 岩土力学, 2016, 37(11): 3309-3316. Zhang Youliang, Tan Fei, Zhang Liren, et al.Scalable parallel computation for finite element model with hundreds of millions of elements in geotechnical engineering[J]. Rock and Soil Mechanics, 2016, 37(11): 3309-3316. [39] Problem 7 asymmetrical conductor with a hole[DB/OL]. https://www.compumag.org/wp/wp-content/uploads/2018/ 06/problem7.pdf.
2022, Vol. 37 
