Abstract：As for the problem that usual optimal power flow algorithm can not meet the timely demand of the complex power grid., this paper presents a novel distributed Q(λ) learning algorithm based on complex districted power grid , which deals no auxiliary process with the optimal power flow (OPF) mathematical model and whose internal agent independently undertakes each district’s learning duty with the standard multi-step Q(λ) learning algorithm, and then coordinately cooperate to reach the optimization of the whole system. The result of the application in IEEE118 bus bar demonstrates that the distributed Q(λ) learning algorithm provides a new feasible and effective method to the complex grid OPF problem.
余涛, 刘靖, 胡细兵. 基于分布式多步回溯Q(λ) 学习的复杂电网最优潮流算法[J]. 电工技术学报, 2012, 27(4): 185-192.
Yu Tao, Liu Jing, Hu Xibing. Optimal Power Flow for Complex Power Grid Using Distributed Multi-Step Backtrack Q(λ) Learning. Transactions of China Electrotechnical Society, 2012, 27(4): 185-192.
[1] David I Sun, Bruce Ashley, Brian Brewer , et al. Optimal power flow by newton approach[J]. IEEE Transactions on Power Apparatus and Systems, 1984, 103(10): 2864-2880. [2] 韦化, 李滨, 杭乃善, 等. 大规模水-火电力系统最优潮流的现代内点算法实现[J]. 中国电机工程学报, 2003, 23(6): 13-18. Wei Hua, Li Bin, Hang Naishan, et al. An implementation of interior point algorithm for large-scale hydro-thermal optimal power flow problems[J]. Proceedings of the CSEE, 2003, 23(6): 13-18. [3] 赵晋泉, 侯志俭, 吴际舜. 改进最优潮流牛顿算法有效性的对策研究[J]. 中国电机工程学报, 1999, 19(12): 70-75. Zhao Jinquan, Hou Zhijian, Wu Jishun. Some new strategies for improving the effectiveness of newton optimal power flow algorithm[J]. Proceedings of the CSEE, 1999, 19(12): 70-75. [4] 周明, 孙树栋. 遗传算法原理及应用[M]. 北京: 国防工业出版社, 1999. [5] Luonan Chen, Hideki Suzuki, Kazuo Katou. Mean field theory for optimal power flow[J]. IEEE Transactions on Power Systems, 1997, 12(4): 1481-1486. [6] 李晓梅, 莫则尧. 可扩展并行算法的设计与分析[M]. 北京: 国防工业出版社, 2000. [7] 潘哲龙, 张伯明, 孙宏斌, 等. 分布计算的遗传算法中无功优化中的应用[J]. 电力系统自动化, 2001, 6(13): 37-41. Pan Zhelong, Zhang Boming, Sun Hongbin et al. A distributid genetic algorithm for reactive power optimization[J]. Automaticon of Electric Power Systems, 2001, 6(13): 37-41. [8] Batut J, Renaud A. Daily generation scheduling optimization with transmission constraints[J]. IEEE Transactions on Power Systems, 2000, 7(3): 982-989. [9] 程新功, 厉吉文, 曹立霞, 等. 电力系统最优潮流的分布式并行算法[J]. 电力系统自动化, 2003, 27(24): 23-27. Cheng Xingong, Li Jiwen, Cao Lixia, et al. Distribution and parallel optimal power flow solution of electric power systems[J]. Automation of Electric Power Systems, 2003, 27(24): 23-27. [10] 李强. 分布式优化算法的算法研究[D]. 北京: 华北电力大学, 2006. [11] 仲宇, 顾国昌, 张汝波. 多智能体系统中的分布式强化学习研究现状[J]. 控制理论与应用, 2003, 20(3): 317-322. Zhong Yu, Gu Guochang, Zhang Rubo. Survey of distributed reinforcement learning algorithms in multi-agent systems[J]. Control Theory & Applications, 2003, 20(3): 317-322. [12] 胥传普, 杨立兵, 刘福斌. 关于节能降耗与电力市场联合实施方案的探讨[J]. 电力系统自动化, 2007, 31(23): 99-103. Xu Chuanpu, Yang Libing, Liu Fubin. Discuss on the Union implementation scheme of energy conservation measures and electricity marketability methods[J]. Automatic of Electric Power Systems, 2007, 31(23): 99-103. [13] Vlachogiannis J G, Hatziagyriou N D. Reinforcement learning for reactive power control[J]. IEEE Transactions on Power Systems, 2004, 19(3): 1317-1325. [14] 邱晓燕, 张子健, 李兴源. 基于改进遗传内点法的电网多目标无功优化[J]. 电网技术, 2009, 33(13): 27-31. Qiu Xiaoyan, Zhang Zijian, Li Xinyuan. Multi- objective reactive power optimization based on improved genetic-interior point algorithm[J]. Power System Technology, 2009, 33(13): 27-31. [15] 仲宇, 顾国昌, 张汝波. 分布式强化学习的体系结构研究[J]. 计算机工程与应用, 2003, 39(11): 111-113. Zhong Yu, Gu Guochang, Zhang Rubo. Research on the architectures of distributed reinforcement learning systems[J]. Computer Engineering and Applications, 2003, 39(11): 111-113. [16] Jing Peng, Williams R J. Incremental multi-step Q-learning[J]. Machine Leaning, 1996(22): 283-290. [17] Watkins J C H, Dayan Peter. Q-learning[J]. Machine Leaning, 1992(8): 279-292. [18] 张汝波. 强化学习理论及应用[M]. 哈尔滨: 哈尔滨工程大学出版社, 2001. [19] Richard S Sutton, Andrew G Barto. Reinforcement learning: an introduction[M]. Cambridge: MIT Press, 1998. [20] 余涛, 周斌, 甄卫国. 强化学习理论在电力系统中的应用及展望[J]. 电力系统保护与控制, 2009, 37(14): 122-128. Yu Tao, Zhou Bin, Zhen Weiguo. Application and development of reinforcement learning theory in power systems[J]. Power System Protection and Control, 2009, 37(14): 122-128. [21] Kim B H, Baldick R. Coarse-grained distributed optimal power flow[J]. IEEE Transactions on Power Systems, 1997, 12(2): 932-939. [22] 刘明波, 谢敏, 赵维兴. 大电网最优潮流计算[M]. 北京: 科学出版社. 2010. [23] Deb K, Pratap A, Agarwal S. A fast and elitist multi-objective genetic algorithm: NSGA-II[J]. IEEE Transactions on Evolutionary Computation, 2002, 6(2): 182-197. [24] H L Liao, Q H Wu, L Jiang. Multi-objective optimization by reinforcement learning for power system dispatch and voltage stability[C]. Proceedings of IEEE PES Conference on Innovative Smart Grid Technologies Europe, Gothenburg, Sweden, 2010: 1-8.
2012, Vol. 27 
