考虑网络安全约束的分布式智能电网边云协同优化调度方法

doi:10.19595/j.cnki.1000-6753.tces.231352

摘要
图/表
参考文献
相关文章 (6)

全文: PDF (1523 KB)
输出: BibTeX | EndNote (RIS)

摘要

随着分布式资源配置容量与电力系统灵活性需求的不断提升,通过分布式智能电网（Distributed Smart Grid, DSG）整合用户侧灵活资源并进行协同调度,对提升分布式电源就地消纳与配电系统实时供需平衡调节能力具有重要意义。考虑到DSG优化过程中的运行经济性、决策生成的实时性与能量网络安全性需求,本文首先将DSG中灵活资源的优化调度过程描述为一个多智能体优化模型,并构建了基于边云协同的DSG系统层级优化调度框架;其次,建立了考虑产消者差异化特征的异构智能体交互环境模型,为兼顾异构产消者的设备运行要求与DSG系统的整体运行经济性与能量网络安全性,设计了考虑全局-局部奖励相结合的产消者智能体奖励方法;最后针对考虑异构智能体的离线训练任务提出了一种改进多智能体近端策略优化算法,并基于IEEE 33节点系统对本文所提方法对DSG系统实时优化调度过程中能量网络安全性、运行经济性与决策时效性的提升作用进行了验证。

	服务

	把本文推荐给朋友
	加入我的书架
	加入引用管理器
	E-mail Alert
	RSS
	作者相关文章

关键词 ：分布式智能电网, 灵活资源, 边云协同, 深度强化学习, 网络安全约束

Abstract：

With the increasing penetration of distributed generation and the growing demand for power system flexibility, issues like voltage rise at the edge of distribution networks and network congestion under bidirectional power flow are becoming more prominent. Integrating and coordinating flexible resources at the user side through Distributed Smart Grids (DSG) is significant for enhancing the accommodation of distributed generation and the real-time supply-demand balancing capability of distribution systems. Considering the large quantity and high dispersion of flexible resource devices and the distinct characteristics of different prosumers, traditional centralized optimization and dispatch schemes as well as distributed computing methods will face greater challenges in solving efficiency and decision delivery timeliness. Against this background, this paper aims to develop a DSG system collaborative optimization and dispatch method that takes into account operational economy, energy network security, and decision timeliness concurrently.
Firstly, mapping real-world prosumers who control and own flexible resources to intelligent agents in reinforcement learning, the optimization and dispatch of flexible resources in DSG is formulated as a multi-agent collaborative optimization model. The existing edge-cloud collaborative framework is extended to the optimization of flexible resources considering energy network security constraints, and a hierarchical optimization and dispatch framework of flexible resource-prosumer-DSG is established. Secondly, considering the differentiated characteristics of prosumers in aspects like types of flexible resource devices, photovoltaics (PV) is taken as distributed generation, and electric vehicles (EV), heating, ventilation and air conditioning (HVAC) of buildings, and energy storage systems (ESS) are taken as demand-side flexible resources. A heterogeneous intelligent agent interactive environment model is built based on the operational characteristics of different flexible resources. Meanwhile, to balance flexible resource operational requirements, overall economic efficiency and energy network security of the DSG system, user satisfaction evaluation of EV and HVAC operation and ESS operation cost are considered as local rewards, while system energy cost and energy network security evaluation are taken as global rewards, and a combined global-local reward mechanism for heterogeneous intelligent agents is proposed. Finally, to adapt to the collaborative training task of the heterogeneous intelligent agent system, an improved multi-agent proximal policy optimization (MAPPO) algorithm is proposed based on asynchronous update of agent policies in random order.
Case studies on the IEEE 33-node system are conducted for analysis. Firstly, the proposed improved MAPPO algorithm is compared with existing multi-agent collaborative training schemes in the offline training stage. Secondly, the differences in flexible resource prosumers' power decisions with and without considering energy network constraints are analyzed in the online dispatch stage. Finally, the proposed method is compared with traditional mathematical programming and particle swarm optimization methods regarding optimization performance in real-time dispatch. The main conclusions are: 1) The edge-cloud collaborative hierarchical optimization and dispatch framework for DSG systems is established, which can obtain dispatch decisions faster in real-time dispatch compared to traditional centralized optimization and thus improve the timeliness of DSG power dispatch decisions. 2) The combined global-local reward mechanism for heterogeneous intelligent agents can achieve overall DSG system optimization and collaborative training objectives of balancing user comfort, economic efficiency and energy network security. 3) The proposed improved MAPPO algorithm adapted for heterogeneous intelligent agent training can maintain independent decision spaces for each agent while ensuring environment state stability in collaborative training through asynchronous policy updates in random order.

Key words： Distributed smart grid flexible resources edge-cloud collaboration deep reinforcement learning network constrain

收稿日期: 2023-08-21

PACS:

TM732

基金资助:

国家自然科学基金面上项目(52177080)和北京市科技新星计划项目(Z201100006820106）资助

通讯作者: 胡俊杰男,1986年生,教授,博士研究生导师,主要研究方向为新能源电力系统及微网。E-mail：junjiehu@ncepu.edu.cn

作者简介: 潘玺安男,1993年生,博士研究生,研究方向为新能源电力系统及微网。E-mail：panxian@ncepu.edu.cn

引用本文:

潘玺安, 艾欣, 胡俊杰, 王坤宇, 王昊洋. 考虑网络安全约束的分布式智能电网边云协同优化调度方法[J]. 电工技术学报, 0, (): 231352-231352. Pan Xi’an, Ai Xin, Hu Junjie, Wang Kunyu, Wang Haoyan. Network Security Constrained Distributed Smart Grid Edge-Cloud Collaborative Optimization Scheduling. Transactions of China Electrotechnical Society, 0, (): 231352-231352.

链接本文:

https://dgjsxb.ces-transaction.com/CN/10.19595/j.cnki.1000-6753.tces.231352 https://dgjsxb.ces-transaction.com/CN/Y0/V/I/231352

[1] 孙毅, 李飞, 胡亚杰, 等. 计及条件风险价值和综合需求响应的产消者能量共享激励策略[J]. 电工技术学报, 2023, 38(9): 2448-2463.
Sun Yi, Li Fei, Hu Yajie, et al.Energy sharing incentive strategy of prosumers considering conditional value at risk and integrated demand response[J]. Transactions of China Electrotechnical Society, 2023, 38(9): 2448-2463.
[2] 胡俊杰, 李阳, 吴界辰, 等. 基于配网节点电价的产消者日前优化调度[J]. 电网技术, 2019, 43(8): 2770-2780.
Hu Junjie, Li Yang, Wu Jiechen, et al.A day-ahead optimization scheduling method for prosumer based on iterative distribution locational marginal price[J]. Power System Technology, 2019, 43(8): 2770-2780.
[3] 李勇, 凌锋, 乔学博, 等. 基于网侧资源协调的自储能柔性互联配电系统日前-日内优化[J]. 电工技术学报, 2023: 8907.
Li Yong, Ling Feng, Qiao Xuebo, et al.Day-Ahead and Intra-Day Optimization of Flexible Interconnected Distribution System with Self-Energy Storage Based on the Grid-Side Resource Coordination[J]. Transactions of China Electrotechnical Society, 2023: 8907.
[4] 赵紫原. 分布式智能电网护航能源安全: 专访中国工程院院士、天津大学教授余贻鑫[J]. 中国电力企业管理, 2022(25): 10-13.
Zhao Ziyuan.Distributed smart grid escorts energy security—interview with yu Yixin, academician of China academy of engineering and professor of Tianjin university[J]. China Power Enterprise Management, 2022(25): 10-13.
[5] 尤毅, 刘东, 钟清, 等. 主动配电网优化调度策略研究[J]. 电力系统自动化, 2014, 38(9): 177-183.
You Yi, Liu Dong, Zhong Qing, et al.Research on optimal schedule strategy for active distribution network[J]. Automation of Electric Power Systems, 2014, 38(9): 177-183.
[6] 王枭, 何怡刚, 马恒瑞, 等. 面向电网辅助服务的虚拟储能电厂分布式优化控制方法[J]. 电力系统自动化, 2022, 46(10): 181-188.
Wang Xiao, He Yigang, Ma Hengrui, et al.Distributed optimization control method of virtual energy storage plants for power grid ancillary services[J]. Automation of Electric Power Systems, 2022, 46(10): 181-188.
[7] 李振坤, 钱晋, 符杨, 等. 基于负荷聚合商优选分级的配电网多重阻塞管理[J]. 电力系统自动化, 2021, 45(19): 109-116.
Li Zhenkun, Qian Jin, Fu Yang, et al.Multiple congestion management for distribution network based on optimization classification of load aggregators[J]. Automation of Electric Power Systems, 2021, 45(19): 109-116.
[8] Hu Junjie, Liu Xuetao, Shahidehpour M, et al.Optimal operation of energy hubs with large-scale distributed energy resources for distribution network congestion management[J]. IEEE Transactions on Sustainable Energy, 2021, 12(3): 1755-1765.
[9] Hu Junjie, Yang Guangya, Bindner H W, et al.Application of network-constrained transactive control to electric vehicle charging for secure grid operation[J]. IEEE Transactions on Sustainable Energy, 2017, 8(2): 505-515.
[10] Hu Junjie, Yang Guangya, Ziras C, et al.Aggregator operation in the balancing market through network-constrained transactive energy[J]. IEEE Transactions on Power Systems, 2019, 34(5): 4071-4080.
[11] Pantoš M.Market-based congestion management in electric power systems with exploitation of aggregators[J]. International Journal of Electrical Power & Energy Systems, 2020, 121: 106101.
[12] Shen Feifan, Wu Qiuwei, Huang Shaojun, et al.Two-tier demand response with flexible demand swap and transactive control for real-time congestion management in distribution networks[J]. International Journal of Electrical Power & Energy Systems, 2020, 114: 105399.
[13] Mnih V, Kavukcuoglu K, Silver D, et al.Human-level control through deep reinforcement learning[J]. Nature, 2015, 518(7540): 529-533.
[14] 陈泽宇, 方志远, 杨瑞鑫, 等. 基于深度强化学习的混合动力汽车能量管理策略[J]. 电工技术学报, 2022, 37(23): 6157-6168.
Chen Zeyu, Fang Zhiyuan, Yang Ruixin, et al.Energy management strategy for hybrid electric vehicle based on the deep reinforcement learning method[J]. Transactions of China Electrotechnical Society, 2022, 37(23): 6157-6168.
[15] Kang H, Jung S, Kim H, et al.Multi-objective sizing and real-time scheduling of battery energy storage in energy-sharing community based on reinforcement learning[J]. Renewable and Sustainable Energy Reviews, 2023, 185: 113655.
[16] 陈明昊, 孙毅, 谢志远. 基于双层深度强化学习的园区综合能源系统多时间尺度优化管理[J]. 电工技术学报, 2023, 38(7): 1864-1881.
Chen Minghao, Sun Yi, Xie Zhiyuan.The multi-time-scale management optimization method for park integrated energy system based on the Bi-layer deep reinforcement learning[J]. Transactions of China Electrotechnical Society, 2023, 38(7): 1864-1881.
[17] 叶宇剑, 袁泉, 刘文雯, 等. 基于参数共享机制多智能体深度强化学习的社区能量管理协同优化[J]. 中国电机工程学报, 2022, 42(21): 7682-7695.
Ye Yujian, Yuan Quan, Liu Wenwen, et al.Parameter sharing empowered multi-agent deep reinforcement learning for coordinated management of energy communities[J]. Proceedings of the CSEE, 2022, 42(21): 7682-7695.
[18] Lin Lin, Guan Xin, Peng Yu, et al.Deep reinforcement learning for economic dispatch of virtual power plant in Internet of energy[J]. IEEE Internet of Things Journal, 2020, 7(7): 6288-6301.
[19] Fang Dawei, Guan Xin, Peng Yu, et al.Distributed deep reinforcement learning for renewable energy accommodation assessment with communication uncertainty in Internet of energy[J]. IEEE Internet of Things Journal, 2021, 8(10): 8557-8569.
[20] Kim D, Bae Y, Yun S, et al.A methodology for generating reduced-order models for large-scale buildings using the Krylov subspace method[J]. Journal of Building Performance Simulation, 2020, 13(4): 419-429.
[21] Chintala R H, Rasmussen B P.Automated multi-zone linear parametric black box modeling approach for building HVAC systems[C]//Proceedings of ASME 2015 Dynamic Systems and Control Conference, Columbus, Ohio, USA, 2016
[22] Silver D, Hubert T, Schrittwieser J, et al.A general reinforcement learning algorithm that Masters chess, shogi, and Go through self-play[J]. Science, 2018, 362(6419): 1140-1144.
[23] Ding Yifu, Morstyn T, McCulloch M D. Distributionally robust joint chance-constrained optimization for networked microgrids considering contingencies and renewable uncertainty[J]. IEEE Transactions on Smart Grid, 2022, 13(3): 2467-2478.
[24] Zhu Hao, Liu H J.Fast local voltage control under limited reactive power: optimality and stability analysis[J]. IEEE Transactions on Power Systems, 2016, 31(5): 3794-3803.
[25] Yu Chao, Velu A, Vinitsky E, et al. The surprising effectiveness of PPO in cooperative, multi-agent games[EB/OL].2021: arXiv: 2103.01955. https://arxiv.org/abs/2103.01955.pdf
[26] Iqbal S, Sha Fei. Actor-attention-critic for multi-agent reinforcement learning[EB/OL].2018: arXiv: 1810.02912. https://arxiv.org/abs/1810.02912.pdf
[27] Foerster J, Farquhar G, Afouras T, et al. Counterfactual multi-agent policy gradients[J]. Proceedings of the AAAI Conference on Artificial Intelligence, 2018, 32(1): - .
[28] Kuba J G, Feng Xidong, Ding Shiyao, et al. Heterogeneous-agent mirror learning: a continuum of solutions to cooperative MARL[EB/OL].2022: arXiv: 2208.01682. https://arxiv.org/abs/2208.01682.pdf
[29] Kuba J G, Chen Ruiqing, Wen Muning, et al. Trust region policy optimisation in multi-agent reinforcement learning[EB/OL].2021: arXiv: 2109.11251. https://arxiv.org/abs/2109.11251.pdf
[30] Wiese F, Schlecht I, Bunke W D, et al.Open Power System Data - Frictionless data for electricity system modelling[J]. Applied Energy, 2019, 236: 401-409.
[31] Liessner R, Schmitt J, Dietermann A, et al.Hyperparameter optimization for deep reinforcement learning in vehicle energy management[C]//Proceedings of the 11th International Conference on Agents and Artificial Intelligence, Prague, Czech Republic, 2019: 134-144.
[32] 马庆, 邓长虹. 基于单/多智能体简化强化学习的电力系统无功电压控制方法[J/OL]. 电工技术学报, 2023: 1-13. (2023-04-19). https://kns.cnki.net/kcms/detail/11.2188.TM.20230419.1028.002.html.
Ma Qing, Deng Changhong. Single/multi agent simplified deep reinforcement learning based volt-var control of power system[J/OL]. Transactions of China Electrotechnical Society, 2023: 1-13. (2023-04-19). https://kns.cnki.net/kcms/detail/11.2188.TM.20230419.1028.002.html.