基于CEQ(<em>λ</em>)多智能体协同学习的互联电网性能标准控制指令动态分配优化算法

摘要
图/表
参考文献
相关文章 (9)

全文: PDF (458 KB)
输出: BibTeX | EndNote (RIS)

摘要在CPS指令动态分配过程中,应用单智能体Q(λ)算法下的CPS合格率虽然可以满足要求,但还存在收敛速度较慢、寻优空间较小等问题。为此,提出应用多智能体相关均衡强化学习的方法来弥补单智能体强化学习的不足。根据调频时延将AGC机组分成煤电、气电和水电等类型,对CEQ(λ)应用于这n类智能体组成的CPS调节指令动态分配进行研究分析。标准两区域模型及南方电网模型仿真研究表明：多智能体CEQ(λ)算法适用于随机、机组组合复杂的互联电网CPS指令动态最优分配,能有效提高系统的适应性、鲁棒性和CPS考核合格率。

	服务

	把本文推荐给朋友
	加入我的书架
	加入引用管理器
	E-mail Alert
	RSS
	作者相关文章
	张孝顺
	余涛
	唐捷

关键词 ：相关均衡, 自动发电控制系统, 协同学习, 强化学习, 随机优化

Abstract：Although automatic generation control under CPS standard can be addressed by classic reinforcement learning Q(λ) algorithm, such problems as slow convergence and small optimal searching space still exist from the view point of multi-agent equilibrium solution. Therefore, this paper proposes correlated-equilibrium-Q(λ) (CEQ(λ)) learning. According to the response time delay of thermal plants, AGC adjustment units are first divided into different kinds of unit, such as coal, gas, hydro and so on. Then dynamic allocation orders of generators are analyzed by CEQ(λ) learning based multi-agent control framework. Simulation tests of two-area load frequency control model and China South Power Grid demonstrate that the CEQ(λ)-learning algorithm is more suitable for CPS instruction dynamic optimal allocation in stochastic and complex interconnection network, and it can enhance the robustness and adaptability of power systems in CPS assessment.

Key words： Correlated-equilibrium automatic generation control system synergetic learning reinforcement learning stochastic optimization

收稿日期: 2014-12-31 出版日期: 2016-04-28

PACS:

TM71

基金资助:国家重点基础研究发展计划（973计划）（2013CB228205）,国家自然科学基金（51177051、51477055）和中国南方电网科技项目资助

通讯作者: 张孝顺男,1990年生,博士研究生,研究方向为人工智能、强化学习、多智能体一致性等算法在电力系统中的应用。E-mail: xszhang1990@sina.con E-mail: taoyul@scut.edu.cn

作者简介: 余涛男,1974年生,教授,博士生导师,研究方向为复杂电力系统的非线性控制理论和仿真。E-mail: taoyul@scut.edu.cn

引用本文:

张孝顺, 余涛, 唐捷. 基于CEQ(λ)多智能体协同学习的互联电网性能标准控制指令动态分配优化算法[J]. 电工技术学报, 2016, 31(8): 125-133. Zhang Xiaoshun, Yu Tao, Tang Jie. Dynamic Optimal Allocation Algorithm for Control Performance Standard Order of Interconnected Power Grids Using Synergetic Learning of Multi-Agent CEQ(λ). Transactions of China Electrotechnical Society, 2016, 31(8): 125-133.

链接本文:

https://dgjsxb.ces-transaction.com/CN/Y2016/V31/I8/125

[1] Jaleeli N, Vanslyck L S. NERC’s new control per- formance standards[J]. IEEE Transactions on Power Systems, 1999, 14(3): 1091-1099.
[2] 唐悦中, 张王俊. 基于CPS的AGC控制策略研究[J]. 电网技术, 2004, 28(21): 75-79.
Tang Yuezhong, Zhang Wangjun. Research on control performance standard based control strategy for AGC[J]. Power System Technology, 2004, 28(21): 75-79.
[3] 高宗和, 滕贤亮, 涂力群. 互联电网AGC分层控制与CPS控制策略[J]. 电力系统自动化, 2004, 28(1): 78-81.
Gao Zonghe, Teng Xianliang, Tu Liqun. Hierarchical AGC mode and CPS control strategy for inter- connected power systems[J]. Automation of Electric Power Systems, 2004, 28(1): 78-81.
[4] 李滨, 韦化, 农蔚涛. 基于现代内点理论的互联电网控制性能评价标准下的AGC控制策略[J]. 中国电机工程学报, 2008, 28(25): 56-61.
Li Bin, Wei Hua, Nong Weitao. AGC control strategy under control performance standard for inter- connected power grid based on optimization theory[J]. Proceedings of the CSEE, 2008, 28(25): 56-61.
[5] 付鹏武, 周念成, 王强钢, 等. 基于时滞模型预测控制算法的网络化AGC研究[J]. 电工技术学报, 2014, 29(4): 188-195.
Fu Pengwu, Zhou Niancheng, Wang Qianggang, et al. Research on networked AGC system based on delay model predictive control algorithm[J]. Transactions of China Electrotechnical Society, 2014, 29(4): 188- 195.
[6] 丁冬, 刘宗歧, 杨水丽, 等. 基于模糊控制的电池储能系统辅助AGC调频方法[J]. 电力系统保护与控制, 2015, 43(8): 81-87.
Ding Dong, Liu Zongqi, Yang Shuili, et al. Battery energy storage aid automatic generation control for load frequency control based on fuzzy control[J]. Power System Protection and Control, 2015, 43(8): 81-87.
[7] Zeynelgil H L, Demiroren A, Sengor N S. The application of ANN technique to automatic gener- ation control for multi-area power system[J]. Inter- national Journal of Electrical Power & Energy Systems, 2002, 24(5): 345-354.
[8] 席磊, 余涛, 张孝顺, 等. 基于狼爬山快速多智能体学习策略的电力系统智能发电控制方法[J]. 电工技术学报, 2015, 30(23): 93-101.
Xi Lei, Yu Tao, Zhang Xiaoshun, et al. A fast multi-agent learning strategy base on DWoLF-PHC(λ) for smart generation control of power systems[J]. Transactions of China Electrotechnical Society, 2015, 30(23): 93-101.
[9] 高宗和. 自动发电控制算法的几点改进[J]. 电力系统自动化, 2001, 25(22): 49-51.
Gao Zonghe. Some algorithmic improvements on AGC software[J]. Automation of Electric Power Systems, 2001, 25(22): 49-51.
[10] 刘斌, 王克英, 余涛, 等. PSO算法在互联电网CPS功率调节中的应用研究[J]. 电力系统保护与控制, 2009, 37(6): 36-39.
Liu Bin, Wang Keying, Yu Tao, et al. Study on the application of particle swarm optimization algorithm to power regulation of CPS in interconnected power grids[J]. Power System Protection and Control, 2009, 37(6): 36-39.
[11] 余涛, 王宇名, 刘前进. 互联电网CPS调节指令动态最优分配Q-学习算法[J]. 中国电机工程学报, 2010, 30(7): 62-69.
Yu Tao, Wang Yuming, Liu Qianjin. Q-learning- based dynamic optimal allocation algorithm for CPS order of interconnected power grids[J]. Proceedings of the CSEE, 2010, 30(7): 62-69.
[12] 余涛, 王宇名, 甄卫国, 等. 基于多步回溯Q学习的自动发电控制指令动态优化分配算法[J]. 控制理论与应用, 2011, 28(1): 58-69.
Yu Tao, Wang Yuming, Zhen Weiguo, et al. Multi-step backtrack Q-learning based dynamic optimal algorithm for auto generation control order dispatch[J]. Control Theory & Applications, 2011, 28(1): 58-69.
[13] 余涛, 王宇名, 叶文加, 等. 基于改进分层强化学习的CPS指令动态优化分配算法[J]. 中国电机工程学报, 2011, 31(19): 90-96.
Yu Tao, Wang Yuming, Ye Wenjia, et al. Multi- objective dynamic optimal dispatch method for cps order of interconnected power grids using improved hierarchical reinforcement learning[J]. Proceedings of the CSEE, 2011, 31(19): 90-96.
[14] Yu T, Wang Y M, Ye W J, et al. Stochastic optimal generation command dispatchbased on improved hierarchical reinforcement learning approach[J]. IET Generation, Transmission & Distribution, 2011, 5(8): 789-797.
[15] Bassar T, Olsder G J. Dynamic non-cooperative game theory[M]. London: SIAM Series in Classics in Applied Mathematics, 1999.
[16] Greenwald A, Hall K, Zinkevich M. Correlated Q-learning[J]. Journal of Machine Learning Research, 2007, 1: 1-30.
[17] Keiding H, Peleg B. Correlated equilibrium of games with many players[J]. International Journal of Game Theory, 2000, 29(3): 375-389.
[18] Littman M. Markov games as a framework for multiagent reinforcement learning[C]//Proceedings of the Eleventh International Conference on Machine Learning, 1994: 157-163.
[19] 刁浩然, 杨明, 陈芳, 等. 基于强化学习理论的地区电网无功电压优化控制方法[J]. 电工技术学报, 2015, 30(12): 408-414.
Diao Haoran, Yang Ming, Chen Fang, et al. Reactive power and voltage optimization control approach of the regional power grid based on reinforcement learning theory[J]. Transactions of China Electro- technical Society, 2015, 30(12): 408-414.
[20] Tao Y, Bin Z, Ka W C, et al. Stochastic optimal relaxed automatic generation control in non-Markov environment based on multi-step Q( λ ) learning[J]. IEEE Transactions on Power Systems, 2011, 26(3): 1272-1282.
[21] 张汝波. 强化学习理论及应用[M]. 哈尔滨: 哈尔滨工程大学出版社, 2001.
[22] Weissgerber J. Dynamic models for steam and hydro turbines in power system studies[J]. IEEE Transa- ctions on Power Apparatus and Systems, 1973, 92(6): 1904-1951.