Reactive power and voltage optimization control approach of the regional power grid based on reinforcement learning theory
Diao Haoran1,Yang Ming1,Chen Fang2,Sun Guozhong3
1. Key Laboratory of Power System Intelligent Dispatch and Control (Shandong University) Ji’nan 250061 China; 2. Automation and Electrical Engineering, University of Ji’nan Ji’nan 250061 China; 3. State Grid Tangshan Electric Power Company Limited Tangshan 063000 China
Abstract：Based on reinforcement learning theory, this paper proposes a practical approach for reactive power and voltage optimization control in regional power grid. The approach uses Q-learning algorithm to learn continuously under interaction between the action policies and grid states, then gets Q value function corresponding to each state - action, and finally forms the optimal grid reactive power and voltage control strategies. The approach gets rid of the convergence problems that existing in traditional reactive power optimization methods for solving nonlinear mixed integer programming model, meanwhile, compared to the multi - zone diagram method, as the Q value function contains global response messages in the whole grid, thus we can comprehensively judge the interactions between each substation and coordinate to control the reactive power and voltage control equipments, then obtain the global optimal control strategies in the jurisdiction grid. The approach paper proposes improves the reactive power and voltage optimization control results. Through a test of an actual 220kV substation and its feeder system, the example demonstrates the effectiveness of the approach.
刁浩然,杨明,陈芳,孙国忠. 基于强化学习理论的地区电网无功电压优化控制方法[J]. 电工技术学报, 2015, 30(12): 408-414.
Diao Haoran,Yang Ming,Chen Fang,Sun Guozhong. Reactive power and voltage optimization control approach of the regional power grid based on reinforcement learning theory. Transactions of China Electrotechnical Society, 2015, 30(12): 408-414.
[1] 刘明波, 陈学军. 基于原对偶仿射尺度内点法的电力系统无功优化算法[J]. 电网技术, 1998, 22(3): 24-28. Liu Mingbo, Chen Xuejun. Prime-dual affine scaling interior point method based reactive power optimiza- tion in power system[J]. Power System Technology, 1998, 22(3): 24-28 [2] 李亚男, 张粒子, 杨以涵. 考虑电压约束裕度的无功优化及其内点解法[J]. 中国电机工程学报, 2001, 21(9): 1-4. Li Yanan, Zhang Lizi, Yang Yihan. Reactive power optimization under voltage constraints margin[J]. Proceedings of the CSEE, 2001, 21(9): 1-4. [3] 许诺, 黄民翔. 原对偶内点法与定界法在无功优化中的应用[J]. 电力系统及其自动化学报, 2000, 12(3): 26-30. Xu Nuo, Huang Minxiang. Application of primal-dual interior point method and branch-bound method in reactive power optimization[J]. Proceedings of the EPSA, 2000, 12(3): 26-30. [4] 刘方, 颜伟. 基于遗传算法和内点法的无功优化混合策略[J]. 中国电机工程学报, 2005, 25(15): 67-72. Liu Fang, Yan Wei. A hybrid strategy based on GA and IPM for optimal reactive power flow[J]. Procee dings of the CSEE, 2005, 25(15): 67-72. [5] 潘珂, 韩学山, 孟祥星. 无功优化内点法中非线性方程组求解规律研究[J]. 电网技术, 2006, 30(19): 59-65. Pan Ke, Han Xueshan, Meng Xiangxing. Solution prin-ciples study of nonlinear correction equations in primal-dual interior point method for reactive power optimization[J]. Power System Technology, 2006, 30(19): 59-65. [6] Suzuki M, Morima E. Coordinated A VQC operations of EHV transformer’s tap changer by fuzzy expert control system[A]. In: 2002 International Conference on Power System Technology[C]. 2002, 1679-1684. [7] 阎振坤, 厉吉文, 李晓华. 基于模糊边界和双九区图的变电站电压无功控制策略研究[J]. 继电器, 2005, 33(10): 36-40. Yan Zhenkun, Li Jiwen, Li Xiaohua. Study of voltage and reactive power integrative control strategy based on fuzzy boundary and double nine-area control method[J]. RELAY, 2005, 33(10): 36-40. [8] 罗毅, 涂光瑜, 金燕云, 等. 基于多区图控制策略的地区电网电压无功优化控制[J]. 继电器, 2004, 32(5): 44-48. Luo Yi, Tu Guangyu, Jin Yanyun, et al. Control over optimization of voltage and reactive power in regional power network based on multi-region chart control strategy[J]. RELAY, 2004, 32(5): 44-48. [9] 余涛, 周斌. 电力系统电压/无功控制策略研究综述[J]. 继电器, 2008, 36(6): 79-85. Yu Tao, Zhou Bin. A survey on voltage/reactive power control strategy for power systems[J]. RELAY, 2008, 36(6): 79-85. [10] Yu T, Zhen W G. A reinforcement learning approach to power system stabilizer[A]. In: Proceedings of the 9th IEEE PES Power & Energy Society General Meeting[C]. 2009: 1-5. [11] 胡细兵. 基于强化学习算法的最优潮流研究[D]. 广州: 华南理工大学, 2011. [12] 袁野. 基于强化学习算法的互联电网AGC随机最优控制[D]. 广州: 华南理工大学, 2011. [13] Vlachogiannis J G, Hatziargyriou N D. Reinforcement learning for reactive power control[J]. IEEE Transac- tions on Power Systems, 2004, 19(3): 1317-1325. [14] Kaelbling L P, Littman M L, Moore A W. Rein- forcement learning: A survey[J]. Journal of Artificial Intelligence Research, 1996: 237-285. [15] 虞靖靓. 基于Q学习的Agent智能决策的研究与实现[D]. 合肥: 合肥工业大学, 2005. [16] 余涛, 周斌, 陈家荣. 基于Q学习的互联电网动态最优CPS控制[J]. 中国电机工程学报, 2009, (19): 13-19. Yu Tao, Zhou Bin, Chen Jiarong. Q-learning based dynamic optimal CPS control methodology for intercon- nected power systems[J]. Proceedings of CSEE, 2009, (19): 13-19. [17] Bertsekas D P, Bertsekas D P. Dynamic programming and optimal control[M]. Belmont, MA: Athena Scientific, 1995. [18] 杜春侠, 高云, 张文. 多智能体系统中具有先验知识的Q学习算法[J]. 清华大学学报: 自然科学版, 2005, (7): 981-984. Du Chunxia, Gao Yun, Zhang Wen. Q-learning with prior knowledge in multi-agent systems[J]. Journal of Tsinghua University (Science and Technology), 2005, (7): 981-984.
2015, Vol. 30 
