|
|
|
| Efficient Deep Deterministic Policy Gradient Algorithm for Real-Time Self-Dispatch of Wind-Storage Power Plant |
| Song Yuhao1, Wei Wei1, Huang Shaowei1, Wu Qiren2, Mei Shengwei1 |
1. Department of Electrical Engineering Tsinghua University Beijing 100084 China;
2. China Three Gorges Renewables (Group) Co. Ltd Beijing 101100 China |
|
|
|
|
Abstract The development of wind power and other renewable energy is of great significance to achieve the dual carbon goal, and the wind-storage power plant is the main form of wind power connected to the power grid in the future. This paper studies the real-time self-dispatch problem of the wind-storage power plant commercialized on the generating side, with the goal of maximizing its expected income. Due to the large prediction error of the field-level wind power and the difficulty in accurately predicting the electricity price of the grid due to the limited information of independent power producers, the real-time self-dispatch of the wind-storage power plant is faced with multiple uncertainties, which is extremely challenging. In this paper, an efficient DDPG algorithm was proposed to solve the real-time self-dispatch strategy of the wind-storage power plant, and realize the field-level online decision-making independent of prediction. Firstly, Lyapunov optimization was used to construct the basic strategy to obtain a good but not necessarily local optimal strategy. Then, samples were pre-generated by the basic strategy to initialize the experience base and improve the search efficiency. Further, DDPG algorithm with expert mechanism was applied to train the locally optimal self-scheduling strategy. Case study shows that compared with the basic dispatch strategy and the classical DDPG, the proposed method can effectively improve the average revenue of the wind-storage power plant.
|
|
Received: 30 May 2022
|
|
|
|
|
|
[1] 国家能源局. 2021年全国电力工业统计数据[EB/OL]. [2022-01-26]. http://www.nea.gov.cn/2022-01/26/c_1310441589.htm.
[2] 国家能源局. 截至3月底全国发电装机容量约24亿千瓦,3月份可再生能源发电量较快增长.[EB/OL]. [2022-04-22]. http://www.nea.gov.cn/2022-04/22/c_1310569074.htm.
[3] 姜书鹏, 乔颖, 徐飞, 等. 风储联合发电系统容量优化配置模型及敏感性分析[J]. 电力系统自动化, 2013, 37(20): 16-21.
Jiang Shupeng, Qiao Ying, Xu Fei, et al.Capacity optimization and sensitivity analysis of cogeneration system of wind power and energy storage[J]. Automation of Electric Power Systems, 2013, 37(20): 16-21.
[4] 陆秋瑜, 罗澍忻, 胡伟, 等. 集群风储联合系统广域协调控制及利益分配策略[J]. 电力系统自动化, 2019, 43(20): 183-191.
Lu Qiuyu, Luo Shuxin, Hu Wei, et al.Wide-area coordinated control and benefit assignment strategy of clustering wind-energy storage integrated system[J]. Automation of Electric Power Systems, 2019, 43(20): 183-191.
[5] 孙辉, 刘鑫, 贲驰, 等. 含风储一体化电站的电力系统多目标风险调度模型[J]. 电力系统自动化, 2018, 42(5): 94-101.
Sun Hui, Liu Xin, Ben Chi, et al.Multi-objective risk scheduling model of power system containing power station with integrated wind power and energy storage[J]. Automation of Electric Power Systems, 2018, 42(5): 94-101.
[6] 王佳丽. 探路风电预报[J]. 能源, 2014(3): 74-76.
Wang Jiali.Pathfinder wind power forecast[J]. Energy, 2014(3): 74-76.
[7] 姚子麟, 张亮, 邹斌, 等. 含高比例风电的电力市场电价预测[J]. 电力系统自动化, 2020, 44(12): 49-55.
Yao Zilin, Zhang Liang, Zou Bin, et al.Electricity price prediction for electricity market with high proportion of wind power[J]. Automation of Electric Power Systems, 2020, 44(12): 49-55.
[8] García C E, Prett D M, Morari M.Model predictive control: theory and practice—a survey[J]. Automatica, 1989, 25(3): 335-348.
[9] Xie Le, Gu Yingzhong, Eskandari A, et al.Fast MPC-based coordination of wind power and battery energy storage systems[J]. Journal of Energy Engineering, 2012, 138(2): 43-53.
[10] Lin Minghong, Liu Zhenhua, Wierman A, et al.Online algorithms for geographical load balancing[C]//2012 International Green Computing Conference (IGCC), San Jose, 2012: 1-10.
[11] Wu Hongyu, Shahidehpour M, Li Zuyi, et al.Chance-constrained day-ahead scheduling in stochastic power system operation[J]. IEEE Transactions on Power Systems, 2014, 29(4): 1583-1591.
[12] Ben-Tal A, Nemirovski A.Robust solutions of uncertain linear programs[J]. Operations Research Letters, 1999, 25(1): 1-13.
[13] Garcia-Gonzalez J, de la Muela R M R, Santos L M, et al. Stochastic joint optimization of wind generation and pumped-storage units in an electricity market[J]. IEEE Transactions on Power Systems, 2008, 23(2): 460-468.
[14] Lara J D, Olivares D E, Cañizares C A.Robust energy management of isolated microgrids[J]. IEEE Systems Journal, 2019, 13(1): 680-691.
[15] Neely M J.Stochastic network optimization with application to communication and queueing systems[M]. Berlin: Springer, 2010.
[16] Guo Zhongjie, Wei Wei, Chen Laijun, et al.Real-time self-dispatch of a remote wind-storage integrated power plant without predictions: explicit policy and performance guarantee[J]. IEEE Open Access Journal of Power and Energy, 2021, 8: 484-496.
[17] 赵冬梅, 陶然, 马泰屹, 等. 基于多智能体深度确定策略梯度算法的有功-无功协调调度模型[J]. 电工技术学报, 2021, 36(9): 1914-1925.
Zhao Dongmei, Tao Ran, Ma Taiyi, et al.Active and reactive power coordinated dispatching based on multi-agent deep deterministic policy gradient algorithm[J]. Transactions of China Electrotechnical Society, 2021, 36(9): 1914-1925.
[18] 李涛, 胡维昊, 李坚, 等. 基于深度强化学习算法的光伏-抽蓄互补系统智能调度[J]. 电工技术学报, 2020, 35(13): 2757-2768.
Li Tao, Hu Weihao, Li Jian, et al.Intelligent economic dispatch for PV-PHS integrated system: a deep reinforcement learning-based approach[J]. Transactions of China Electrotechnical Society, 2020, 35(13): 2757-2768.
[19] 刁浩然, 杨明, 陈芳, 等. 基于强化学习理论的地区电网无功电压优化控制方法[J]. 电工技术学报, 2015, 30(12): 408-414.
Diao Haoran, Yang Ming, Chen Fang, et al.Reactive power and voltage optimization control approach of the regional power grid based on reinforcement learning theory[J]. Transactions of China Electrotechnical Society, 2015, 30(12): 408-414.
[20] 梁煜东, 陈峦, 张国洲, 等. 基于深度强化学习的多能互补发电系统负荷频率控制策略[J]. 电工技术学报, 2022, 37(7): 1768-1779.
Liang Yudong, Chen Luan, Zhang Guozhou, et al.Load frequency control strategy of hybrid power generation system: a deep reinforcement learning—based approach[J]. Transactions of China Electrotechnical Society, 2022, 37(7): 1768-1779.
[21] 于一潇, 杨佳峻, 杨明, 等. 基于深度强化学习的风电场储能系统预测决策一体化调度[J]. 电力系统自动化, 2021, 45(1): 132-140.
Yu Yixiao, Yang Jiajun, Yang Ming, et al.Prediction and decision integrated scheduling of energy storage system in wind farm based on deep reinforcement learning[J]. Automation of Electric Power Systems, 2021, 45(1): 132-140.
[22] Lillicrap T P, Hunt J J, Pritzel A, et al. Continuous control with deep reinforcement learning[EB/OL].2019: arXiv: 1509.02971. https://arxiv.org/abs/1509.02971
[23] 蔡新雷, 崔艳林, 董锴, 等. 基于改进K-means和MADDPG算法的风储联合系统日前优化调度方法[J]. 储能科学与技术, 2021, 10(6): 2200-2208.
Cai Xinlei, Cui Yanlin, Dong Kai, et al.Day-ahead optimal scheduling approach of wind-storage joint system based on improved K-means and MADDPG algorithm[J]. Energy Storage Science and Technology, 2021, 10(6): 2200-2208.
[24] 张淑兴, 马驰, 杨志学, 等. 基于深度确定性策略梯度算法的风光储系统联合调度策略[J/OL]. 中国电力, 2022: 1-9. [2022-07-03]. http://kns.cnki.net/kcms/detail/11.3265.TM.20211115.1426.002.html.
Zhang Shuxing, Ma Chi, Yang Zhixue, et al. Joint dispatch of wind-photovoltaic-storage hybrid system based on deep deterministic policy gradient algorithm [J/OL]. Electric Power, 2022: 1-9. [2022-07-03]. http://kns.cnki.net/kcms/detail/11.3265.TM.20211115.1426.002.html.
[25] Shen Ziqi, Wei Wei, Wu Danman, et al.Modeling arbitrage of an energy storage unit without binary variables[J]. CSEE Journal of Power and Energy Systems, 2021, 7(1): 156-161.
[26] 束金龙, 闻人凯. 线性规划理论与模型应用[M]. 北京: 科学出版社, 2003.
[27] Sigaud O, Buffet O.Markov decision processes in artificial intelligence[M]. New York: John Wiley & Sons, 2013. |
|
|
|
2022, Vol. 37 
