结合数据驱动与物理模型的主动配电网双时间尺度电压协调优化控制

doi:10.19595/j.cnki.1000-6753.tces.222273

摘要
图/表
参考文献
相关文章 (15)

全文: PDF (12329 KB)
输出: BibTeX | EndNote (RIS)

摘要

高比例电动汽车、分布式风电、光伏接入配电网,导致电压频繁、快速、剧烈波动。传统调压设备与逆变器动作速度差异巨大,如何协调是难点问题。本文结合数据驱动与物理建模方法,提出了一种配电网双时间尺度电压协调优化控制策略。针对短时间尺度（分钟级）电压波动,以静止无功补偿器、分布式电源无功为决策变量,以电压平方偏差最小为目标函数,针对平衡与不平衡配电网,基于支路潮流方程,计及物理约束构建了二次规划模型。针对长时间尺度（小时级）电压波动,以电压调节器变比、可投切电容电抗器档位、储能系统充放电功率为动作,当前时段配电网节点功率为状态,节点电压平方偏差为代价,构建了马尔可夫决策过程。为克服连续-离散动作空间维数灾, 提出了一种基于松弛-预报-校正的深度确定性策略梯度强化学习求解算法。最后,采用IEEE 33节点平衡与123节点不平衡配电网验证了所提出方法的性能。

	服务

	把本文推荐给朋友
	加入我的书架
	加入引用管理器
	E-mail Alert
	RSS
	作者相关文章
	张剑
	崔明建
	何怡刚

关键词 ：智能配电网, 电压控制, 深度强化学习, 二次规划, 双时间尺度

Abstract：

A large number of electric vehicles (EVs), distributed solar and/or wind turbine generators (WTGs) connected to distribution systems lead to frequent and sharp voltages fluctuations. The action rates of conventional adjustable devices and smart inverters are very different. In this context, a novel dual-timescale voltage control scheme is proposed by organically combining data-driven with physics-based optimization. On fast timescale, a quadratic programming (QP) for balanced and unbalanced distribution systems is developed based on branch flow equations. The optimal reactive power of renewable distributed generators (DGs) and static VAR compensators (SVCs) is configured on several minutes. Whereas, on slow timescale, a data-driven Markovian decision process (MDP) is developed, in which the charge/discharge power of energy storage systems (ESSs), statuses/ratios of switchable capacitors reactors (SCRs), and voltage regulators (VRs) are configured hourly to minimize long-term discounted squared voltages magnitudes deviations using an adapted deep deterministic policy gradient (DDPG) deep reinforcement learning (DRL) algorithm. The capabilities of the proposed method are validated with IEEE 33-bus balanced and 123-bus unbalanced distribution systems.
The contributions of this paper are summarized as follows: (1)Combining data-driven with physics-based methods, a strategy for coordinated control of five different types of adjustable equipment, namely VRs, SCRs, ESSs, SVCs and DGs inverters on fast and slow timescales is proposed. (2) A slow timescale (say 1 hour) MDP for active and reactive power coordination is constructed. The (near) optimal settings of ratios/statuses of VRs, SCRs and charge/discharge power of ESSs are found using DRL algorithm. As a result, the deficiency of low computing rate for conventional physics -based large scale mixed integer non-convex nonlinear stochastic programming is completely overcome. (3) The charge/discharge power of ESS is continuous variable while ratios/statues of VRs and SCRs are discrete decisions. DDPG algorithm cannot be directly applicable to discrete action while DQN algorithm cannot be applicable to continuous action. Further, when there are a large number of VRs and SCRs in distribution network, DQN algorithm leads to dimensionality curses. The existing DRL algorithm in literatures cannot deal with joint continuous-discrete action (efficiently). To eliminate dimensionality curses in joint continuous- discrete action space, ratios/statuses of VRs and SCRs are firstly relaxed to continuous variables. Then, for the proto action given by actor of DDPG agent, ${{K}_{\text{nn}}}$ nearest neighbors are found out in the joint continuous-discrete action space. Finally, each of the ${{K}_{\text{nn}}}$ actions is transferred to the critic of DDPG agent one by one to evaluate its value. The action with the greatest value is chosen to interact with the distribution network. (4) Given the (near) optimal solution of the slow timescale MDP, a QP for VVO with DGs, SVCs inverters settings on fast timescale (say several minutes or seconds) to minimize squared voltages magnitudes deviations are developed for balanced and unbalanced distribution systems. As a result, voltage violations on fast timescale caused by sizable, rapid and frequent power fluctuations from renewable DGs and fast charged EVs can be mitigated in real time. (5) One of the outstanding advantages of the proposed method is very easy to perform in practice with near optimal solution. Further, when ${{K}_{\text{nn}}}$=20 or 40, the proposed method has much more stable training process than multi-agent DQN algorithm and much higher computing rate than conventional multi-slot single fast timescale mixed integer QP by about 18.0~36.7 times.

Key words： smart distribution systems voltage control DRL QP dual-timescale

收稿日期: 2022-12-12

PACS:

TM732

基金资助:

国家自然科学基金资助项目（52207130）

通讯作者: 张剑 1982年生,男,博士,讲师,研究方向为电力系统建模、主动配电网技术、电动汽车有序充电等。E-mail：z_jj1219@sina.com

作者简介: 崔明建 1987年生,,男,博士,教授,研究方向为风力预测、机组组合、配电网物理信息系统等。E-mail：mingjian.cui@ieee.org。

引用本文:

张剑, 崔明建, 何怡刚. 结合数据驱动与物理模型的主动配电网双时间尺度电压协调优化控制[J]. 电工技术学报, 0, (): 222273-. Zhang Jian, Cui Mingjian, He Yigang. Dual Timescales Coordinated and Optimal Voltages Control in Distribution Systems using Data-Driven and Physical Optimization. Transactions of China Electrotechnical Society, 0, (): 222273-.

链接本文:

https://dgjsxb.ces-transaction.com/CN/10.19595/j.cnki.1000-6753.tces.222273 https://dgjsxb.ces-transaction.com/CN/Y0/V/I/222273

[1] Yong Jiaying, Ramachandaramurthy V K, Tan Kangmiao, et al.Bi-directional electric vehicle fast charging station with novel reactive power compensation for voltage regulation[J]. International Journal of Electrical Power & Energy Systems, 2015, 64: 300-310.
[2] Wang Pengfei, Liang D H, Yi Jialiang, et al.Integrating electrical energy storage into coordinated voltage control schemes for distribution networks[J]. IEEE Transactions on Smart Grid, 2014, 5(2): 1018-1032.
[3] Ranamuka D, Agalgaonkar A P, Muttaqi K M.Examining the interactions between DG units and voltage regulating devices for effective voltage control in distribution systems[J]. IEEE Transactions on Industry Applications, 2017, 53(2): 1485-1496.
[4] Agalgaonkar Y P, Pal B C, Jabr R A.Distribution voltage control considering the impact of PV generation on tap changers and autonomous regulators[J]. IEEE Transactions on Power Systems, 2014, 29(1): 182-192.
[5] Kawabe K, Tanaka K.Impact of dynamic behavior of photovoltaic power generation systems on short-term voltage stability[J]. IEEE Transactions on Power Systems, 2015, 30(6): 3416-3424.
[6] Taranto G N, Assis T M L, Falcao D M, et al. Highlighting the importance of chronology on voltage protection and control in active distribution networks[J]. IEEE Transactions on Power Delivery, 2017, 32(1): 361-369.
[7] Bose S, Low S H, Teeraratkul T, et al.Equivalent relaxations of optimal power flow[J]. IEEE Transactions on Automatic Control, 2015, 60(3): 729-742.
[8] Valverde G, Van Cutsem T.Model predictive control of voltages in active distribution networks[J]. IEEE Transactions on Smart Grid, 2013, 4(4): 2152-2161.
[9] 黄大为, 王孝泉, 于娜, 等. 计及光伏出力不确定性的配电网混合时间尺度无功/电压控制策略[J]. 电工技术学报, 2022, 37(17): 4377-4389.
Huang Dawei, Wang Xiaoquan, Yu Na, et al.Hybrid timescale voltage/var control in distribution network considering PV power uncertainty[J]. Transactions of China Electrotechnical Society, 2022, 37(17): 4377-4389.
[10] Magnússon S, Qu Guannan, Fischione C, et al.Voltage control using limited communication[J]. IEEE Transactions on Control of Network Systems, 2019, 6(3): 993-1003.
[11] Bolognani S, Carli R, Cavraro G, et al.Distributed reactive power feedback control for voltage regulation and loss minimization[J]. IEEE Transactions on Automatic Control, 2015, 60(4): 966-981.
[12] 杨珺, 侯俊浩, 刘亚威, 等. 分布式协同控制方法及在电力系统中的应用综述[J]. 电工技术学报, 2021, 36(19): 4035-4049.
Yang Jun, Hou Junhao, Liu Yawei, et al.Distributed cooperative control method and application in power system[J]. Transactions of China Electrotechnical Society, 2021, 36(19): 4035-4049.
[13] 李忠文, 程志平, 张书源, 等. 考虑经济调度及电压恢复的直流微电网分布式二次控制[J]. 电工技术学报, 2021, 36(21): 4482-4492.
Li Zhongwen, Cheng Zhiping, Zhang Shuyuan, et al.Distributed secondary control for economic dispatch and voltage restoration of DC microgrid[J]. Transactions of China Electrotechnical Society, 2021, 36(21): 4482-4492.
[14] Zhang Liang, Wang Gang, Giannakis G B.Real-time power system state estimation and forecasting via deep unrolled neural networks[J]. IEEE Transactions on Signal Processing, 2019, 67(15): 4069-4077.
[15] 巨云涛, 陈希, 李嘉伟, 等. 基于分布式深度强化学习的微网群有功无功协调优化调度[J]. 电力系统自动化, 2023, 47(1): 115-125.
Ju Yuntao, Chen Xi, Li Jiawei, et al.Active and reactive power coordinated optimal dispatch of networked microgrids based on distributed deep reinforcement learning[J]. Automation of Electric Power Systems, 2023, 47(1): 115-125.
[16] 顾雪平, 刘彤, 李少岩, 等. 基于改进双延迟深度确定性策略梯度算法的电网有功安全校正控制[J]. 电工技术学报, 2023, 38(8): 2162-2177.
Gu Xueping, Liu Tong, Li Shaoyan, et al.Active power correction control of power grid based on improved twin delayed deep deterministic policy gradient algorithm[J]. Transactions of China Electrotechnical Society, 2023, 38(8): 2162-2177.
[17] 陈泽宇, 方志远, 杨瑞鑫, 等. 基于深度强化学习的混合动力汽车能量管理策略[J]. 电工技术学报, 2022, 37(23): 6157-6168.
Chen Zeyu, Fang Zhiyuan, Yang Ruixin, et al.Energy management strategy for hybrid electric vehicle based on the deep reinforcement learning method[J]. Transactions of China Electrotechnical Society, 2022, 37(23): 6157-6168.
[18] 陈明昊, 孙毅, 谢志远. 基于双层深度强化学习的园区综合能源系统多时间尺度优化管理[J]. 电工技术学报, 2023, 38(7): 1864-1881.
Chen Minghao, Sun Yi, Xie Zhiyuan.The multi-time-scale management optimization method for park integrated energy system based on the Bi-layer deep reinforcement learning[J]. Transactions of China Electrotechnical Society, 2023, 38(7): 1864-1881.
[19] 宋煜浩, 魏韡, 黄少伟, 等. 风储联合电站实时自调度的高效深度确定性策略梯度算法[J]. 电工技术学报, 2022, 37(23): 5987-5999.
Song Yuhao, Wei Wei, Huang Shaowei, et al.Efficient deep deterministic policy gradient algorithm for real-time self-dispatch of wind-storage power plant[J]. Transactions of China Electrotechnical Society, 2022, 37(23): 5987-5999.
[20] Xu Hanchen, Domínguez-García A D, Sauer P W. Optimal tap setting of voltage regulation transformers using batch reinforcement learning[J]. IEEE Transactions on Power Systems, 2020, 35(3): 1990-2001.
[21] Al-Saffar M, Musilek P.Reinforcement learning-based distributed BESS management for mitigating overvoltage issues in systems with high PV penetration[J]. IEEE Transactions on Smart Grid, 2020, 11(4): 2980-2994.
[22] Wang Wei, Yu Nanpeng, Gao Yuanqi, et al.Safe off-policy deep reinforcement learning algorithm for volt-VAR control in power distribution systems[J]. IEEE Transactions on Smart Grid, 2020, 11(4): 3008-3018.
[23] Cao Di, Hu Weihao, Zhao Junbo, et al.A multi-agent deep reinforcement learning based voltage regulation using coordinated PV inverters[J]. IEEE Transactions on Power Systems, 2020, 35(5): 4120-4123.
[24] Yang Qiuling, Wang Gang, Sadeghi A, et al.Two-timescale voltage control in distribution grids using deep reinforcement learning[J]. IEEE Transactions on Smart Grid, 2020, 11(3): 2313-2323.
[25] Zhang Ying, Wang Xinan, Wang Jianhui, et al.Deep reinforcement learning based volt-VAR optimization in smart distribution systems[J]. IEEE Transactions on Smart Grid, 2021, 12(1): 361-371.
[26] Zhang Jian, He Yigang.Fast solving method for two-stage multi-period robust optimization of active and reactive power coordination in active distribution networks[J]. IEEE Access, 2022, 11: 30208-30222.
[27] Ding Tao, Liu Shiyu, Yuan Wei, et al.A two-stage robust reactive power optimization considering uncertain wind power integration in active distribution networks[J]. IEEE Transactions on Sustainable Energy, 2016, 7(1): 301-311.