SOC Prediction of Lithium-Ion Batteries Based on Sequence-to-Sequence Model with Adversarial Weighted Attention Mechanism
Chen Zhiming, Liu Jianhua, Ke Tianci, Chen Kewei
1. School of Computer Science and Mathematics Fujian University of Technology Fuzhou 350018 China; 2. Fujian Provincial Key Laboratory of Big Data Mining and Applications Fuzhou 350018 China
Abstract：In practical use, lithium-ion batteries often face extreme temperatures, and the behavior of different battery systems varies under different discharge conditions. Traditional state of charge (SOC) prediction methods, based on physical models, often overlook these differences, resulting in significant prediction errors. In recent years, as deep learning techniques, including recurrent neural networks (RNNs) and Transformers, have gained attention in time series forecasting, most of these methods have shown limited progress in improving prediction accuracy. To address these challenges, this paper introduces an adversarial weighted attention sequence-to-sequence (AWAS) model to enhance SOC prediction. First, features are extracted from the lithium-ion battery dataset to create an input matrix. This matrix is then sent through an encoder with gated recurrent units (GRU) to capture feature correlations. Next, on top of the multi-head attention mechanism, an additional linear layer is introduced to compute a weight matrix W related to the number of attention heads. The output of this linear layer matches the number of attention heads, facilitating extra linear transformations of queries, keys, and values, and mapping them into the weight space of the multi-head attention. This enhances the computational flexibility of the attention mechanism. Then, the relevant information extracted by the GRU is input into the improved weight attention mechanism and is then passed to the GRU in the decoder. This process strengthens the extraction of correlated information among the features. Finally, the concept of adversarial training is introduced, using a three-layer convolutional layer as the core of the discriminator. The output of the decoder's GRU is considered as sequence 1, while the corresponding SOC values for the given time period are treated as sequence 2. The truthfulness of sequence 1 and sequence 2 is evaluated using the sigmoid function. Adversarial training mitigates the issue of gradient vanishing in the GRU, resolving long-term dependencies. Ultimately, the encoder's output is processed through a fully connected layer to obtain the SOC prediction. The results show that for single-step prediction tasks, our proposed model achieved significantly reduced root mean square error (RMSE) and mean absolute error (MAE) on the LG-HG2 dataset, with values of 0.141 2% and 0.109 4% respectively. On the Panasonic dataset, the errors further decreased to 0.101 3% and 0.080 3%. The Sparse Informer model, which outperformed others in control experiments, achieved errors of 0.260 2%, 0.218 2%, 0.380 1%, and 0.278 2%. In the case of a 12-step prediction task, our model achieved the lowest average MAE of 0.108 7% and mean absolute percentage error (MAPE) of 0.173 4% on the Panasonic dataset at -20℃.The results of ablation experiments indicated that the average MAPE for 12-step predictions decreased to 0.347 2%, while the average RMSE and MAE for single-step predictions reduced to 0.096 6% and 0.083 7%, respectively. These findings validate the effectiveness of our model architecture. The experiments lead to these conclusions: (1) Compared to numerous RNN and transformer models, the incorporation of adversarial training into the Seq2Seq model effectively mitigates the issues of gradient vanishing and exploding. Across various prediction tasks, the error evaluations for the three different datasets consistently yielded the lowest values. Thus, the introduction of adversarial training is deemed appropriate. (2) Conventional multi-head attention mechanisms exhibit a substantial increase in MAPE as the lithium battery temperature rises. In contrast, the proposed weight attention mechanism reduces errors and curbs the upward trend in errors. From the experimental results, it is evident that the weight attention mechanism is more practical.
陈治铭, 刘建华, 柯添赐, 陈可纬. 基于对抗性的权重注意力机制序列到序列模型的锂离子电池SOC估计方法[J]. 电工技术学报, 2024, 39(19): 6244-6256.
Chen Zhiming, Liu Jianhua, Ke Tianci, Chen Kewei. SOC Prediction of Lithium-Ion Batteries Based on Sequence-to-Sequence Model with Adversarial Weighted Attention Mechanism. Transactions of China Electrotechnical Society, 2024, 39(19): 6244-6256.
[1] 高德欣, 郑晓雨, 王义, 等. 电动汽车充电状态监测与多级安全预报警方法[J]. 电工技术学报, 2022, 37(9): 2252-2262. Gao Dexin, Zheng Xiaoyu, Wang Yi, et al.A state monitoring and multi-level safety pre-warning method for electric vehicle charging process[J]. Transactions of China Electrotechnical Society, 2022, 37(9): 2252-2262. [2] 武龙星, 庞辉, 晋佳敏, 等. 基于电化学模型的锂离子电池荷电状态估计方法综述[J]. 电工技术学报, 2022, 37(7): 1703-1725. Wu Longxing, Pang Hui, Jin Jiamin, et al.A review of SOC estimation methods for lithium-ion batteries based on electrochemical model[J]. Transactions of China Electrotechnical Society, 2022, 37(7): 1703-1725. [3] 王义军, 左雪. 锂离子电池荷电状态估算方法及其应用场景综述[J]. 电力系统自动化, 2022, 46(14): 193-207. Wang Yijun, Zuo Xue.Review on estimation methods for state of charge of lithium-ion battery and their application scenarios[J]. Automation of Electric Power Systems, 2022, 46(14): 193-207. [4] Zhou Wenlu, Zheng Yanping, Pan Zhengjun, et al.Review on the battery model and SOC estimation method[J]. Processes, 2021, 9(9): 1685. [5] Al Hadi A M R, Ekaputri C, Reza M. Estimating the state of charge on lead acid battery using the open circuit voltage method[J]. Journal of Physics: Conference Series, 2019, 1367(1): 012077. [6] Zhang Mingyue, Fan Xiaobin.Design of battery management system based on improved ampere-hour integration method[J]. International Journal of Electric and Hybrid Vehicles, 2022, 14(1/2): 1. [7] Masmoudi A, Hamdi J, Hadrich Belguith L.Deep learning for sentiment analysis of Tunisian dialect[J]. Computación y Sistemas, 2021, 25(1): 129-148. [8] 刘素贞, 袁路航, 张闯, 等. 基于超声时域特征及随机森林的磷酸铁锂电池荷电状态估计[J]. 电工技术学报, 2022, 37(22): 5872-5885. Liu Suzhen, Yuan Luhang, Zhang Chuang, et al.State of charge estimation of LiFeO4 batteries based on time domain features of ultrasonic waves and random forest[J]. Transactions of China Electrotechnical Society, 2022, 37(22): 5872-5885. [9] 李超然, 肖飞, 樊亚翔, 等. 基于门控循环单元神经网络和Huber-M估计鲁棒卡尔曼滤波融合方法的锂离子电池荷电状态估算方法[J]. 电工技术学报, 2020, 35(9): 2051-2062. Li Chaoran, Xiao Fei, Fan Yaxiang, et al.A hybrid approach to lithium-ion battery SOC estimation based on recurrent neural network with gated recurrent unit and Huber-M robust Kalman filter[J]. Transactions of China Electrotechnical Society, 2020, 35(9): 2051-2062. [10] Wei Meng, Ye Min, Li Jiabo, et al.State of charge estimation of lithium-ion batteries using LSTM and NARX neural networks[J]. IEEE Access, 2020, 8: 189236-189245. [11] 潘锦业, 王苗苗, 阚威, 等. 基于Adam优化算法和长短期记忆神经网络的锂离子电池荷电状态估计方法[J]. 电气技术, 2022, 23(4): 25-30. Pan Jinye, Wang Miaomiao, Kan Wei, et al.State of charge estimation of lithium-ion battery based on Adam optimization algorithm and long short-term memory neural network[J]. Electrical Engineering, 2022, 23(4): 25-30. [12] 李宁, 何复兴, 马文涛, 等. 基于经验模态分解的门控循环单元神经网络的锂离子电池荷电状态估计[J]. 电工技术学报, 2022, 37(17): 4528-4536. Li Ning, He Fuxing, Ma Wentao, et al.State-of-charge estimation of lithium-ion battery based on gated recurrent unit using empirical mode decomposition[J]. Transactions of China Electrotechnical Society, 2022, 37(17): 4528-4536. [13] Jiao Meng, Wang Dongqing, Qiu Jianlong.A GRU-RNN based momentum optimized algorithm for SOC estimation[J]. Journal of Power Sources, 2020, 459: 228051. [14] Zhou Haoyi, Zhang Shanghang, Peng Jieqi, et al.Informer: beyond efficient transformer for long sequence time-series forecasting[J]. Proceedings of the AAAI Conference on Artificial Intelligence, 2021, 35(12): 11106-11115. [15] Liu Xiaowei, Li Kenli, Li Keqin.Attentive semantic and perceptual faces completion using self-attention generative adversarial networks[J]. Neural Processing Letters, 2020, 51(1): 211-229. [16] 何滢婕, 刘月峰, 边浩东, 等. 基于Informer的电池荷电状态估算及其稀疏优化方法[J]. 电子学报, 2023, 51(1): 50-56. He Yingjie, Liu Yuefeng, Bian Haodong, et al.State-of-charge estimation of lithium-ion battery based on Informer and its sparse optimization method[J]. Acta Electronica Sinica, 2023, 51(1): 50-56. [17] Luo Tao, Cao Xudong, Li Jin, et al.Multi-task prediction model based on ConvLSTM and encoder-decoder[J]. Intelligent Data Analysis, 2021, 25(2): 359-382. [18] Liu Di, Li Qiang, Li Sen, et al.Non-autoregressive sparse transformer networks for pedestrian trajectory prediction[J]. Applied Sciences, 2023, 13(5): 3296. [19] Abumohsen M, Owda A Y, Owda M.Electrical load forecasting using LSTM, GRU, and RNN algorithms[J]. Energies, 2023, 16(5): 2283. [20] Wang Jinrui, Han Baokun, Bao Huaiqian, et al.Data augment method for machine fault diagnosis using conditional generative adversarial networks[J]. Proceedings of the Institution of Mechanical Engineers, Part D: Journal of Automobile Engineering, 2020, 234(12): 2719-2727. [21] Chemali E, Kollmeyer P J, Preindl M, et al.Long short-term memory networks for accurate state-of-charge estimation of Li-ion batteries[J]. IEEE Transactions on Industrial Electronics, 2018, 65(8): 6730-6739. [22] Vidal C, Kollmeyer P, Chemali E, et al.Li-ion battery state of charge estimation using long short-term memory recurrent neural network with transfer learning[C]//2019 IEEE Transportation Electrification Conference and Expo (ITEC), Detroit, MI, USA, 2019: 1-6. [23] Xing Yinjiao, He Wei, Pecht M, et al.State of charge estimation of lithium-ion batteries using the open-circuit voltage at various ambient temperatures[J]. Applied Energy, 2014, 113: 106-115. [24] Nakamura K, Hong B W.Adaptive weight decay for deep neural networks[J]. IEEE Access, 2019, 7: 118857-118865.
2024, Vol. 39 
