Intelligent Economic Dispatch for PV-PHS Integrated System: a Deep Reinforcement Learning-Based Approach
Li Tao1, Hu Weihao1, Li Jian1, Han Xiaoyan2, Chen Zhe3
1. School of Mechanical and Electrical Engineering University of Electronic Science and Technology of China Chengdu 611731 China 2. State Grid Sichuan Electric Power Company Chengdu 611041 China 3. Department of Energy Technology Aalborg University Pontoppidanstraede 111 Aalborg Denmark
Abstract：It is important for power system stability and economic operation to effectively suppress the power fluctuation from the large-scale grid-connected photovoltaic (PV) stations on the point of common coupling (PCC). Energy storage system (ESS) may effectively provide the power support to smooth the PV output power to the main grid. However, there are some drawbacks for PV power forecast, such as low accuracy, the frequently charging and discharging exchange of ESS, which may result in low stability and economic benefits. Based on this motivation, the modified cycling decay learning rate- deep deterministic policy gradient (CDLR-DDPG) approach is proposed in this paper, to implement the online intelligent economic dispatch for the PV-PHS complementary power generation system considering both the power fluctuation on PCC and the promise of full absorption of PV. The Markov decision process is introduce to convert this dispatch model and the CDLR-DDPG algorithm is adopted to solve it. Finally, a case study is carried out to evaluate the performance of intelligent economic dispatch model based on the real PV plant obtained in Xiaojin County, Sichuan province, China. The simulation results reveal that the intelligent dispatch strategy can effectively mitigate the power fluctuation and enhance the economic efficiency, that is, the power fluctuation on PCC is reduced by 12.7% and the economic revenue of complementary system is increased by 4.95%, simultaneously.
李涛, 胡维昊, 李坚, 韩晓言, 陈哲. 基于深度强化学习算法的光伏-抽蓄互补系统智能调度[J]. 电工技术学报, 2020, 35(13): 2757-2768.
Li Tao, Hu Weihao, Li Jian, Han Xiaoyan, Chen Zhe. Intelligent Economic Dispatch for PV-PHS Integrated System: a Deep Reinforcement Learning-Based Approach. Transactions of China Electrotechnical Society, 2020, 35(13): 2757-2768.
[1] 太阳能发展“十三五”规划[EB/OL]. 国家能源局, 2016-12-08.http://zfxxgk.nea.gov.cn/auto87/201612/ t20161216_2358.htm. [2] 2018年光伏发电统计信息[EB/OL]. 国家能源局,2019-03-19.http://www.nea.gov.cn/2019-03/19/c_ 137907428.htm. [3] 孙玉树, 张国伟, 唐西胜, 等. 风电功率波动平抑下的MPC双储能控制策略研究[J]. 电工技术学报, 2019, 34(3): 571-578. Sun Yushu, Zhang Guowei, Tang Xisheng, et al.Research on MPC and dual energy storage control strategies with wind power fluctuation mitigation[J]. Transactions of China Electrotechnical Society, 2019, 34(3): 571-578. [4] 李建林, 牛萌, 周喜超, 等. 能源互联网中微能源系统储能容量规划及投资效益分析[J]. 电工技术学报, 2020, 35(4): 874-884. Li Jianlin, Niu Meng, Zhou Xichao, et al.Energy storage capacity planning and investment benefit analysis of micro-energy system in energy interconn-ection[J]. Transactions of China Electrotechnical Society, 2020, 35(4): 874-884. [5] Shah R, Mithulananthan N, Lee K Y.Large-scale PV plant with a robust controller considering power oscillation damping[J]. IEEE Transactions on Energy Conversion, 2013, 28(1): 106-116. [6] Miao Zhixin, Fan Lingling, Osborn D, et al.Control of DFIG-based wind generation to improve interarea oscillation damping[J]. IEEE Transactions on Energy Conversion, 2009, 24(2): 415-422. [7] Zhu Yongli, Liu Chengxi, Sun Kai, et al.Optimization of battery energy storage to improve power system oscillation damping[J]. IEEE Transactions on Sustainable Energy, 2019, 10(3): 1015-1024. [8] 熊连松, 修连成, 王慧敏, 等. 储能系统抑制电网功率振荡的机理研究[J]. 电工技术学报, 2019, 34(20): 4373-4380. Xiong Liansong, Xiu Liancheng, Wang Huimin, et al.Mechanism of energy storage system to suppress grid power oscillations[J]. Transactions of China Electrotechnical Society, 2019, 34(20): 4373-4380. [9] Sukumar S, Mokhlis H, Mekhilef S, et al.Ramp-rate control approach based on dynamic smoothing parameter to mitigate solar PV output fluctuations[J]. International Journal of Electrical Power & Energy Systems, 2018, 96: 296-305. [10] Velasquez A, Barreiro-Gomez J, Quijano N, et al.Distributed model predictive control for economic dispatch of power systems with high penetration of renewable energy resources[J]. International Journal of Electrical Power & Energy Systems, 2019, 113: 607-617. [11] Moarefdoost M, Lamadrid J, Zuluaga L.A robust model for the ramp-constrained economic dispatch problem with uncertain renewable energy[J]. Energy Economics, 2016, 56: 310-325. [12] Nge C, Ranaweera I, Midtgard O, et al.A real-time energy management system for smart grid integrated photovoltaic generation with battery storage[J]. Renewable Energy, 2019, 130: 774-785. [13] Banshwar A, Sharma N, Sood Y, et al.Market-based participation of energy storage scheme to support renewable energy sources for the procurement of energy and spinning reserve[J]. Renewable Energy, 2019, 135: 326-344. [14] 李佳琪, 陈健, 张文, 等. 高渗透率光伏配电网中电池储能系统综合运行控制策略[J]. 电工技术学报, 2019, 34(2): 437-446. Li Jiaqi, Chen Jian, Zhang Wen, et al.Integrated control strategy for battery energy storage systems in distribution networks with high photovoltaic penetration[J]. Transactions of China Electrotechnical Society, 2019, 34(2): 437-446. [15] Zhao Shuqiang, Fang Yuchen, Wei Ziyu.Stochastic optimal dispatch of integrating concentrating solar power plants with wind farms[J]. International Journal of Electrical Power & Energy Systems, 2019, 109: 575-583. [16] Yin Yue, Liu Tianqi, He Chuan.Day-ahead stochastic coordinated scheduling for thermal-hydro-wind- photovoltaic systems[J]. Energy, 2019, 187: 115944. [17] Hang S, Jiakun F, Xiaomeng A, et al.Stochastic optimization of economic dispatch for microgrid based on approximate dynamic programming[J]. IEEE Transactions on Smart Grid, 2019, 10(3): 2440-52. [18] Sutton R S, Barto A G.Reinforcement learning: an introduction[M]. Cambridge: The MIT Press, 1998. [19] 杨挺, 赵黎媛, 王成山. 人工智能在电力系统及综合能源系统中的应用综述[J]. 电力系统自动化, 2019, 43(1): 2-14. Yang Ting, Zhao Liyuan, Wang Chengshan.Review on application of artificial intelligence in power system and integrated energy system[J]. Automation of Electric Power Systems, 2019, 43(1): 2-14. [20] 安灵旭, 唐其筠, 李中成, 等. 人工智能在配电网运维中的应用研究[J]. 电气技术, 2019, 20(10): 103-106. An Lingxu, Tang Qijun, Li Zhongcheng, et al.Research on the application of artificial intelligence in operation maintenance of distribution system[J]. Electrical Engineering, 2019, 20(10): 103-106. [21] Wei Qinglai, Liu Derong, Shi Guang, A novel dual iterative Q-learning method for optimal battery management in smart residential environments[J]. IEEE Transactions on Industrial Electronics, 2015 62(4): 2509-2518. [22] Qiao Junfei, Wang Gongming, Li Wenjing, et al.An adaptive deep Q-learning strategy for handwritten digit recognition[J]. Neural Networks, 2018, 107: 61-71. [23] Wu Jingda, He Hongwen, Peng Jiankun, et al.Continuous reinforcement learning of energy management with deep Q network for a power split hybrid electric bus[J]. Applied Energy, 2018, 222: 799-811. [24] Hua Haochen, Qin Yuchao, Hao Chuantong, et al.Optimal energy management strategies for energy Internet via deep reinforcement learning approach[J]. Applied Energy, 2019, 239: 598-609. [25] Mocanu E, Mocanu D C, Nguyen P H, et al.On-line building energy optimization using deep reinforcement learning[J]. IEEE Transactions on Smart Grid, 2019, 10(4): 3698-3708. [26] Ardizzon G, Cavazzini G, Pavesi G.A new generation of small hydro and pumped-hydro power plants: advances and future challenges[J]. Renewable and Sustainable Energy Reviews, 2014, 31: 746-761. [27] Jordan M I, Mitchell T M.Machine learning: trends, perspectives, and prospects[J]. Science, 2015, 349: 255-260. [28] Olivier S, Olivier B.Markov decision process in artificial intelligence[M]. Piscataway: John Wiley & Sons, Inc, 2013. [29] 最终决战！OpenAI将挑战Dota2最强战队TI8冠军OG[OL]. 新浪体育, 2019-03-27, http://sports. sina.com. cn/go/2019-03-27/doc-ihtxyzsm0868576.shtml. [30] Aneke M, Wang Meihong.Energy storage technologies and real-life applications - a state of the art review[J]. Applied Energy, 2016, 179: 350-377. [31] Yalmip R20190425 < https://yalmip.github.io/ >[Z]. [32] 赖昌伟, 黎静华, 陈博, 等. 光伏发电出力预测技术研究综述[J]. 电工技术学报, 2019, 34(6): 1201-1217. Lai Changwei, Li Jinghua, Chen Bo, et al.Review of photovoltaic power output prediction technology[J]. Transactions of China Electrotechnical Society, 2019, 34(6): 1201-1217. [33] Das U K, Tey K S, Mehdi S, et al.Forecasting of photovoltaic power generation and model optimization: a review[J]. Renewable and Sustainable Energy Reviews, 2018, 81: 912-928. [34] Rana M, Koprinska I, Agelidis V G.Forecasting solar power generated by grid connected PV systems using ensembles of neural networks[C]//2015 International Joint Conference on Neural Networks (IJCNN), Killarney, 2015: 1-8.
2020, Vol. 35 
