基于CEQ(<em>λ</em>)多智能体协同学习的互联电网性能标准控制指令动态分配优化算法

Abstract
Figure/Table
References
Related Citation (9)

Download: PDF (458 KB) HTML (1 KB)
Export: BibTeX | EndNote (RIS)

Abstract Although automatic generation control under CPS standard can be addressed by classic reinforcement learning Q(λ) algorithm, such problems as slow convergence and small optimal searching space still exist from the view point of multi-agent equilibrium solution. Therefore, this paper proposes correlated-equilibrium-Q(λ) (CEQ(λ)) learning. According to the response time delay of thermal plants, AGC adjustment units are first divided into different kinds of unit, such as coal, gas, hydro and so on. Then dynamic allocation orders of generators are analyzed by CEQ(λ) learning based multi-agent control framework. Simulation tests of two-area load frequency control model and China South Power Grid demonstrate that the CEQ(λ)-learning algorithm is more suitable for CPS instruction dynamic optimal allocation in stochastic and complex interconnection network, and it can enhance the robustness and adaptability of power systems in CPS assessment.

Key words： Correlated-equilibrium automatic generation control system synergetic learning reinforcement learning stochastic optimization

Received: 31 December 2014 Published: 28 April 2016

PACS:

TM71

	Service

	E-mail this article
	Add to my bookshelf
	Add to citation manager
	E-mail Alert
	RSS
	Articles by authors
	Zhang Xiaoshun
	Yu Tao
	Tang Jie

Cite this article:

Zhang Xiaoshun,Yu Tao,Tang Jie. Dynamic Optimal Allocation Algorithm for Control Performance Standard Order of Interconnected Power Grids Using Synergetic Learning of Multi-Agent CEQ(λ)[J]. Transactions of China Electrotechnical Society, 2016, 31(8): 125-133.

URL:

https://dgjsxb.ces-transaction.com/EN/ OR https://dgjsxb.ces-transaction.com/EN/Y2016/V31/I8/125

[1] Jaleeli N, Vanslyck L S. NERC’s new control per- formance standards[J]. IEEE Transactions on Power Systems, 1999, 14(3): 1091-1099.
[2] 唐悦中, 张王俊. 基于CPS的AGC控制策略研究[J]. 电网技术, 2004, 28(21): 75-79.
Tang Yuezhong, Zhang Wangjun. Research on control performance standard based control strategy for AGC[J]. Power System Technology, 2004, 28(21): 75-79.
[3] 高宗和, 滕贤亮, 涂力群. 互联电网AGC分层控制与CPS控制策略[J]. 电力系统自动化, 2004, 28(1): 78-81.
Gao Zonghe, Teng Xianliang, Tu Liqun. Hierarchical AGC mode and CPS control strategy for inter- connected power systems[J]. Automation of Electric Power Systems, 2004, 28(1): 78-81.
[4] 李滨, 韦化, 农蔚涛. 基于现代内点理论的互联电网控制性能评价标准下的AGC控制策略[J]. 中国电机工程学报, 2008, 28(25): 56-61.
Li Bin, Wei Hua, Nong Weitao. AGC control strategy under control performance standard for inter- connected power grid based on optimization theory[J]. Proceedings of the CSEE, 2008, 28(25): 56-61.
[5] 付鹏武, 周念成, 王强钢, 等. 基于时滞模型预测控制算法的网络化AGC研究[J]. 电工技术学报, 2014, 29(4): 188-195.
Fu Pengwu, Zhou Niancheng, Wang Qianggang, et al. Research on networked AGC system based on delay model predictive control algorithm[J]. Transactions of China Electrotechnical Society, 2014, 29(4): 188- 195.
[6] 丁冬, 刘宗歧, 杨水丽, 等. 基于模糊控制的电池储能系统辅助AGC调频方法[J]. 电力系统保护与控制, 2015, 43(8): 81-87.
Ding Dong, Liu Zongqi, Yang Shuili, et al. Battery energy storage aid automatic generation control for load frequency control based on fuzzy control[J]. Power System Protection and Control, 2015, 43(8): 81-87.
[7] Zeynelgil H L, Demiroren A, Sengor N S. The application of ANN technique to automatic gener- ation control for multi-area power system[J]. Inter- national Journal of Electrical Power & Energy Systems, 2002, 24(5): 345-354.
[8] 席磊, 余涛, 张孝顺, 等. 基于狼爬山快速多智能体学习策略的电力系统智能发电控制方法[J]. 电工技术学报, 2015, 30(23): 93-101.
Xi Lei, Yu Tao, Zhang Xiaoshun, et al. A fast multi-agent learning strategy base on DWoLF-PHC(λ) for smart generation control of power systems[J]. Transactions of China Electrotechnical Society, 2015, 30(23): 93-101.
[9] 高宗和. 自动发电控制算法的几点改进[J]. 电力系统自动化, 2001, 25(22): 49-51.
Gao Zonghe. Some algorithmic improvements on AGC software[J]. Automation of Electric Power Systems, 2001, 25(22): 49-51.
[10] 刘斌, 王克英, 余涛, 等. PSO算法在互联电网CPS功率调节中的应用研究[J]. 电力系统保护与控制, 2009, 37(6): 36-39.
Liu Bin, Wang Keying, Yu Tao, et al. Study on the application of particle swarm optimization algorithm to power regulation of CPS in interconnected power grids[J]. Power System Protection and Control, 2009, 37(6): 36-39.
[11] 余涛, 王宇名, 刘前进. 互联电网CPS调节指令动态最优分配Q-学习算法[J]. 中国电机工程学报, 2010, 30(7): 62-69.
Yu Tao, Wang Yuming, Liu Qianjin. Q-learning- based dynamic optimal allocation algorithm for CPS order of interconnected power grids[J]. Proceedings of the CSEE, 2010, 30(7): 62-69.
[12] 余涛, 王宇名, 甄卫国, 等. 基于多步回溯Q学习的自动发电控制指令动态优化分配算法[J]. 控制理论与应用, 2011, 28(1): 58-69.
Yu Tao, Wang Yuming, Zhen Weiguo, et al. Multi-step backtrack Q-learning based dynamic optimal algorithm for auto generation control order dispatch[J]. Control Theory & Applications, 2011, 28(1): 58-69.
[13] 余涛, 王宇名, 叶文加, 等. 基于改进分层强化学习的CPS指令动态优化分配算法[J]. 中国电机工程学报, 2011, 31(19): 90-96.
Yu Tao, Wang Yuming, Ye Wenjia, et al. Multi- objective dynamic optimal dispatch method for cps order of interconnected power grids using improved hierarchical reinforcement learning[J]. Proceedings of the CSEE, 2011, 31(19): 90-96.
[14] Yu T, Wang Y M, Ye W J, et al. Stochastic optimal generation command dispatchbased on improved hierarchical reinforcement learning approach[J]. IET Generation, Transmission & Distribution, 2011, 5(8): 789-797.
[15] Bassar T, Olsder G J. Dynamic non-cooperative game theory[M]. London: SIAM Series in Classics in Applied Mathematics, 1999.
[16] Greenwald A, Hall K, Zinkevich M. Correlated Q-learning[J]. Journal of Machine Learning Research, 2007, 1: 1-30.
[17] Keiding H, Peleg B. Correlated equilibrium of games with many players[J]. International Journal of Game Theory, 2000, 29(3): 375-389.
[18] Littman M. Markov games as a framework for multiagent reinforcement learning[C]//Proceedings of the Eleventh International Conference on Machine Learning, 1994: 157-163.
[19] 刁浩然, 杨明, 陈芳, 等. 基于强化学习理论的地区电网无功电压优化控制方法[J]. 电工技术学报, 2015, 30(12): 408-414.
Diao Haoran, Yang Ming, Chen Fang, et al. Reactive power and voltage optimization control approach of the regional power grid based on reinforcement learning theory[J]. Transactions of China Electro- technical Society, 2015, 30(12): 408-414.
[20] Tao Y, Bin Z, Ka W C, et al. Stochastic optimal relaxed automatic generation control in non-Markov environment based on multi-step Q( λ ) learning[J]. IEEE Transactions on Power Systems, 2011, 26(3): 1272-1282.
[21] 张汝波. 强化学习理论及应用[M]. 哈尔滨: 哈尔滨工程大学出版社, 2001.
[22] Weissgerber J. Dynamic models for steam and hydro turbines in power system studies[J]. IEEE Transa- ctions on Power Apparatus and Systems, 1973, 92(6): 1904-1951.