Abstract：Energy storage hybrid trams use the energy storage system as the only power source, and optimize the design of energy management strategy, which can improve the running performance and economic benefits of the tram. Regarding the demand power of the tram as a Markov process, and in order to avoid the impact on the energy management strategy when the driving conditions change greatly, a reinforcement learning energy management strategy based on the recognition of the operating conditions is proposed. Based on historical driving data, the tram driving conditions are constructed and the Markov power state transition matrix under different operating conditions is obtained. Then, with the goal of minimizing the energy consumption of the hybrid energy storage system, the power allocation strategy under different working conditions is obtained through the reinforcement learning algorithm. Finally, the improved learning vector quantization (LVQ) neural network is used to recognize the current driving conditions in real time, and the control system makes real-time decisions based on the current recognized conditions and train status. Real vehicle data is used for simulation verification. The optimized strategy can reduce the energy storage system loss and can be applied to different driving conditions. Experimental verification with a 90kW hybrid energy storage platform verifies the feasibility of the strategy in practical engineering applications.
莫浩楠, 杨中平, 林飞, 王玙, 安星锟. 有轨电车基于工况识别的强化学习能量管理策略[J]. 电工技术学报, 2021, 36(19): 4170-4182.
Mo Haonan, Yang Zhongping, Lin Fei, Wang Yu, An Xingkun. Reinforcement Learning Energy Management Strategy of Tram Based on Condition Identification. Transactions of China Electrotechnical Society, 2021, 36(19): 4170-4182.
[1] 姜军. 有轨电车的复兴与思考[J]. 江苏城市规划, 2014(5): 44-47. Jiang Jun.The revival and reflection of trams[J]. Jiangsu City Planning, 2014(5): 44-47. [2] 王健全, 袁富卫. 城市有轨电车供电方式探讨[J]. 电力机车与城轨车辆, 2015, 38(1):47-51, 58. Wang Jianquan, Yuan Fuwei.Discussion on power supply method of urban tram[J]. Electric Locomotive and Urban Rail Vehicle, 2015, 38(1):47-51, 58. [3] 诸斐琴, 杨中平, 林飞, 等. 基于加速时间预测的现代有轨电车储能系统能量管理与容量配置优化研究[J]. 电工技术学报, 2017, 32(23):158-166. Zhu Feiqin, Yang Zhongping, Lin Fei, et al.Research on energy management and capacity allocation optimization of modern tram energy storage system based on acceleration time prediction[J]. Transactions of China Electrotechnical Society, 2017, 32(23): 158-166. [4] 王玙, 杨中平, 李峰, 等. 有轨电车混合动力系统能量交互型管理策略与容量配置协同优化研究[J]. 电工技术学报, 2019, 34(8):1780-1788. Wang Yu, Yang Zhongping, Li Feng, et al.Study on energy interactive management strategy and capacity allocation collaborative optimization of tram hybrid power system[J]. Transactions of China Electrote- chnical Society, 2019, 34(8):1780-1788. [5] 张弛. 现代有轨电车车载混合储能系统能量管理优化研究[D]. 北京: 北京交通大学, 2018. [6] 韦绍远, 姜久春, 程龙, 等. 储能式有轨电车车地一体化配置模型[J]. 电工技术学报, 2019, 34(2): 427-436. Wei Shaoyuan, Jiang Jiuchun, Cheng Long, et al.Ground-storage model of energy-storage trams[J]. Transactions of China Electrotechnical Society, 2019, 34(2): 427-436. [7] 周美兰, 冯继峰, 张宇. 纯电动汽车复合储能系统及其能量控制策略[J]. 电机与控制学报, 2019, 23(5): 52-59. Zhou Meilan, Feng Jifeng, Zhang Yu.Compound energy storage system of pure electric vehicle and its energy control strategy[J]. Electric Machines and Control, 2019, 23(5): 52-59. [8] 安星锟, 杨中平, 王玙, 等. 基于改进型凸优化算法的有轨电车混合储能系统容量配置帕累托解集[J]. 电工技术学报, 2020, 35(14):3116-3125. An Xingkun, Yang Zhongping, Wang Yu, et al.Pareto solution set of capacity configuration of tram hybrid energy storage system based on improved convex optimization algorithm[J]. Transactions of China Electrotechnical Society, 2020, 35(14): 3116-3125. [9] 王玙, 杨中平, 林飞, 等. 有轨电车车载混合储能系统动态比例分配策略[J]. 电工技术学报, 2019, 34(增刊1):405-413. Wang Yu, Yang Zhongping, Lin Fei, et al.Dynamic proportional distribution strategy for on-board hybrid energy storage system of trams[J]. Transactions of China Electrotechnical Society, 2019, 34(S1): 405-413. [10] 李峰, 杨中平, 王玙, 等. 基于庞特里亚金极小值原理的混合储能有轨电车能量管理策略[J]. 电工技术学报, 2019, 34(增刊2):752-759. Li Feng, Yang Zhongping, Wang Yu, et al.Energy management strategy of hybrid energy storage tram based on Pontrya’s gold minimum principle[J]. Transactions of China Electrotechnical Society, 2019, 34(S2):752-759. [11] 林歆悠, 孙冬野. 基于工况识别的混联式混合动力客车控制策略研究[J]. 中国机械工程, 2012, 23(7): 869-874. Lin Xinyou, Sun Dongye.Research on control strategy of hybrid electric bus based on condition identification[J]. China Mechanical Engineering, 2012, 23(7): 869-874. [12] 罗少华. 基于工况识别的混联式混合动力汽车能量管理策略研究[D]. 重庆: 重庆大学, 2016. [13] Changle Xiang, Feng Ding, Weida Wang, et al.Energy management of a dual-mode power-split hybrid electric vehicle based on velocity prediction and nonlinear model predictive control[J]. Applied Energy, 2017, 189: 640-653. [14] 张洁丽. 基于模型预测控制的插电式混合动力客车能量管理策略研究[D]. 北京: 北京理工大学,2016. [15] Chen Zheng, Hu Hengjie, Wu Yitao, et al.Energy management for a power-split plug-in hybrid electric vehicle based on reinforcement learning[J]. Applied Sciences, 2018, 8(12): 2494. [16] Liu Teng, Zou Yuan, Liu Dexing, et al.Reinforcement learning-based energy management strategy for a hybrid electric tracked vehicle[J]. Energies, 2015, 8(7): 7243-7260. [17] Liu Teng, Hu Xiaosong, Hu Weihao, et al.A heuristic planning reinforcement learning-based energy management for power-split plug-in hybrid electric vehicles[J]. IEEE Transactions on Industrial Informatics, 2019, 15(12): 6436-6445. [18] Liu Teng, Zou Yuan, Liu Dexing, et al.Reinforcement learning of adaptive energy management with transition probability for a hybrid electric tracked vehicle[J]. IEEE Transactions on Industrial Electronics, 2015, 62(12): 7837-7846. [19] Cao Jiayi, Xiong Rui.Reinforcement learning-based real-time energy management for plug-in hybrid electric vehicle with hybrid energy storage system[J]. Energy Procedia, 2017, 142: 1895-1901. [20] Rui Xiong, Cao Jiayi, Yu Quanqing.Reinforcement learning-based real-time power management for hybrid energy storage system in the plug-in hybrid electric vehicle[J]. Applied Energy, 2018, 211: 538-548. [21] Liu Teng, Wang Bo, Yang Chenglang.Online Markov chain-based energy management for a hybrid tracked vehicle with speedy Q-learning[J]. Energy, 2018, 160: 544-555.
2021, Vol. 36 
