Abstract This paper proposes a multi-agent (MA) smart generation control scheme for the coordination of automatic generation control (AGC) in the power grid with system uncertainties.A novel MA new algorithm,i.e.DWoLF-PHC(λ) with a multi-step backtracking and a variable learning rate,is developed,which can effectively identify the optimal average policies under various operating conditions by the control performance standard (CPS).Based on the mixed strategy and the average policy,the algorithm is highly adaptive in stochastic Non-Markov environments and large time-delay systems and can also achieve AGC coordination in interconnected complex power systems in the presence of increasing penetration of renewable energies.Simulation studies on both a two-area load-frequency control (LFC) power system and the China Southern Power Grid model have been done respectively.The results show that the algorithm can achieve the optimal average policies,the closed-loop system has excellent properties,and the algorithm has a fast convergence rate and a higher learning ability compared with other existing intelligent methods.
Xi Lei,Yu Tao,Zhang Xiaoshun等. A Fast Multi-agent Learning Strategy Base on DWoLF-PHC(λ) for Smart Generation Control of Power Systems[J]. Transactions of China Electrotechnical Society, 2015, 30(23): 93-101.
[1] 温步瀛.计及调速器死区影响的两区域互联电力系统AGC研究[J].电工技术学报,2010,25(9):176-182. Wen Buying.Research on AGC of two-area interconnected power system considering the effect of the governor dead band[J].Transactions of China Electrotechnical Society,2010,25(9):176-182. [2] 付鹏武,周念成,王强钢,等.基于时滞模型预测控制算法的网络化AGC研究[J].电工技术学报,2014,29(4):188-195. Fu Pengwu,Zhou Niancheng,Wang Qianggang,et al.Research on networked AGC system based on delay model predictive control algorithm[J].Transactions of China Electrotechnical Society,2014,29(4):188-195. [3] 赵旋宇.南方电网直调机组AGC研究[J].电力系统保护与控制,2008,36(7):54-58. Zhao Xuanyu.Research on AGC for generations directly controlled by CSG[J].Power System Protection and Control,2008,36(7):54-58. [4] Pandey S K,Mohanty S R,Kishor N.A literature survey on load-frequency control for conventional and distribution generation power systems[J].Renewable and Sustainable Energy Reviews,2013,25(5):318-334. [5] Oneal A R.A simple method for improving control area performance:area control error (ACE) diversity interchange[J].IEEE Transactions on Power Systems,1995,10(2):1071-1076. [6] 杜贵和,王正风.智能电网调度一体化设计与研究[J].电力系统保护与控制,2010,38(15):127-131. Du Guihe,Wang Zhengfeng.Design and research on power network dispatching integration of smart grid[J].Power System Protection and Control,2010,38(15):127-131. [7] 吴国沛,刘育权.智能配电网技术支持系统的研究与应用[J].电力系统保护与控制,2010,38(21):162-166,172. Wu Guopei,Liu Yuquan.Research and application of technology support system for smart distribute grid[J].Power System Protection and Control,2010,38(21):162-166,172. [8] 尹明,王成山,葛旭波,等.中德风电发展的比较与分析[J].电工技术学报,2010,25(9):157-163. Yin Ming,Wang Chengshan,Ge Xubo,et al.Comparison and analysis of wind power development between China and Germany[J].Transactions of China Electrotechnical Society,2010,25(9):157-163. [9] Daneshfar F,Bevrani H.Load-frequency control:a GA-based multi-agent reinforcement learning[J].IET Generation Transmission & Distribution,2010,4(1):13-26. [10]Bevrani H,Daneshfar F,Hiyama T.A new intelligent agent-based AGC design with real-time application[J].IEEE Transactions on Systems,Man,and Cybernetics,Part C:Applications and Reviews,2012,42(6):994-1002. [11]Littman M.A generalized reinforcement-learning model:convergence and applications[C].Proceedings of the 13th International Conference on Machine Learning,Bari,Italy,1996:310-318. [12]Hu J,Wellman M P.Multiagent reinforcement learning:Theoretical framework and an algorithm[C].Proceedings of 15th International Conference on Machine Learning,Madison,1999:242-250. [13]Littman M L.Friend or foe Q-learning in general-sum Markov games[C].Proceedings of the 18th International Conference on Machine Learning,Williamstown,Massachusetts,2001:322-328. [14]Yu Tao,Xi Lei,Yang Bo,et al.Multi-agent stochastic dynamic game for smart generation control[J].Journal of Energy Engineering,2015,DOI:10.1061/(ASCE) EY.1943-7897.0000275:04015012. [15]Bowling M,Veloso M.Multiagent learning using a variable learning rate[J].Artificial Intelligence,2002,136(2):215-250. [16]Peng Jing,Williams R J.Incremental multi-step Q-learning[J].Machine Learning,1996,22(1-3):283-290. [17]Sutton R S,Barto A G.Reinforcement Learning:An Introduction[M].Cambridge:MIT Press,1998. [18]Watkins C J H,Dayan P.Q-learning[J].Machine Learning,1992,8(3/4):279-292. [19]Sutton R S.Learning to predict by the methods of temporal differences[J].Machine Learning,1988,3(1):9-44. [20]Yu T,Zhou B,Chan K W,et al.Stochastic optimal relaxed automatic generation control in Non-Markov environment based on multi-step Q(λ) learning[J].IEEE Transactions on Power Systems,2011,26(3):1272-1282. [21]Elgerd O I.Electric Energy System Theory-An Introduction[M].2nd ed.New York:McGraw-Hill,1982. [22]Ernst D,Glavic M,Wehenkel L.Power systems stability control:reinforcement learning framework[J].IEEE Transactions on Power Systems,2004,19(1):427-435. [23]Yu T,Zhou B,Chan K W,et al.R(λ) imitation learning for automatic generation control of interconnected power grids[J].Automatica,2012,48(9):2130-2136. [24]Xi Lei,Yu Tao,Yang Bo,et al.A novel multi-agent decentralized win or learn fast policy hill-climbing with eligibility trace algorithm for smart generation control of interconnected complex power grids[J].Energy Conversion and Management,2015,103(10):82-93.
2015, Vol. 30 
