基于在线附加Q学习的伺服电机速度最优跟踪控制方法

doi:10.19595/j.cnki.1000-6753.tces.L80703

摘要
图/表
参考文献
相关文章 (15)

全文: PDF (1084 KB)
输出: BibTeX | EndNote (RIS)

摘要该文将在线Q学习方法与附加控制思想相结合,讨论了其在伺服系统中电机速度最优跟踪控制问题上的应用。首先在线性二次型跟踪器问题的框架下对待求解问题进行了定义;然后给出了在线附加Q学习迭代式地进行策略评价、策略改善的具体算法。仿真测试中,首先为电机速度跟踪问题设计了传统的PI控制器,然后将基于该文思路所设计的附加控制器与其并联,组成新的速度控制器。仿真结果表明,附加控制器显著改善了电机速度跟踪的动态响应特性,并且具备在被控系统参数发生改变时自动调优的自适应能力。非线性系统在特定条件下可进行局部线性化时,也可用该方法来得到更优的控制性能。

	服务

	把本文推荐给朋友
	加入我的书架
	加入引用管理器
	E-mail Alert
	RSS
	作者相关文章
	邹晓敏
	肖曦
	何琪
	ShkodyrevVyacheslav

关键词 ：在线Q学习, 最优跟踪控制, 附加控制, 电机控制, 速度控制

Abstract：This paper combined online Q-learning with supplementary control and discussed its application to the optimal tracking control problem of servo motor speed. Firstly, the problem to be solved was defined in the framework of linear quadratic tracking. Then, the iterative algorithm of policy evaluation and policy improvement for online supplementary Q-learning was given. In the simulation test, for the motor speed tracking problem in servo system, the traditional PI controller was firstly designed, and then the supplementary controller proposed in this paper was connected to it in parallel to form a new speed controller. The simulation results showed that the additional controller significantly improves the dynamic response characteristics of motor speed tracking, and has the adaptive ability to automatically adjust when parameters of the controlled system changes. When the nonlinear system can be locally linearized under certain conditions, the proposed method can also be applied to obtain better control performance.

Key words： Online Q-learning optimal tracking control supplementary control electric machine control speed control

收稿日期: 2017-11-30 出版日期: 2019-03-21

PACS:

TM301.2

基金资助:国家自然科学基金（51577095）和清华大学自主科研计划支持项目资助

通讯作者: 肖曦, 男,1973 年生,教授,博士生导师,研究方向为高性能伺服电机控制、机器人驱动控制技术、海浪发电技术、电力储能与微电网技术等。E-mail：xiao_xi@tsinghua.edu.cn

作者简介: 邹晓敏,男,1993年生,硕士研究生,研究方向为半实物仿真系统、自适应动态规划、电机控制等。E-mail：zouxm16@mails.tsinghua.edu.cn

引用本文:

邹晓敏,肖曦,何琪,ShkodyrevVyacheslav. 基于在线附加Q学习的伺服电机速度最优跟踪控制方法[J]. 电工技术学报, 2019, 34(5): 917-923. Zou Xiaomin, Xiao Xi, He Qi, Shkodyrev Vyacheslav. Optimal Tracking Control of Servo Motor Speed Based on Online Supplementary Q-Learning. Transactions of China Electrotechnical Society, 2019, 34(5): 917-923.

链接本文:

https://dgjsxb.ces-transaction.com/CN/10.19595/j.cnki.1000-6753.tces.L80703 https://dgjsxb.ces-transaction.com/CN/Y2019/V34/I5/917

[1] Lewis F L, Vrabie D L, Syrmos V L.Optimal control[M]. 3rd ed. New York: John Wiley & Sons, Inc., 2012.
[2] Lewis F L, Vrabie D.Reinforcement learning and adaptive dynamic programming for feedback control[J]. IEEE Circuits & Systems Magazine, 2009, 9(3): 32-50.
[3] 张化光, 张欣, 罗艳红, 等. 自适应动态规划综述[J]. 自动化学报, 2013, 39(4): 303-311.
Zhang Huaguang, Zhang Xin, Luo Yanhong, et al.An overview of research on adaptive dynamic programming[J]. Acta Automatica Sinica, 2013, 39(4): 303-311.
[4] Kiumarsi B, Lewis F L, Modares H, et al.Reinforcement Q-learning for optimal tracking control of linear discrete-time systems with unknown dynamics[J]. Automatica, 2014, 50(4): 1167-1175.
[5] Wang Ding, Liu Derong, Wei Qinglai.Finite-horizon neuro-optimal tracking control for a class of discrete-time nonlinear systems using adaptive dynamic programming approach[J]. Neurocomputing, 2012, 78(1): 14-22.
[6] Guo Wentao, Liu Feng, Si Jennie, et al.Online supplementary ADP learning controller design and application to power system frequency control with large-scale wind energy integration[J]. IEEE Transactions on Neural Networks and Learning Systems, 2016, 27(8): 1748-1761.
[7] Guo Wentao, Liu Feng, Si Jenjie, et al.Approximate dynamic programming based supplementary reactive power control for DFIG wind farm to enhance power system stability[J]. Neurocomputing, 2015, 170: 417-427.
[8] Malla N, Tamrakar U, Shrestha D, et al.Online learning control for harmonics reduction based on current controlled voltage source power inverters[J]. IEEE/CAA Journal of Automatica Sinica, 2017, 4(3): 447-457.
[9] Malla N, Shrestha D, Ni Zhen, et al.Supplementary control for virtual synchronous machine based on adaptive dynamic programming[C]//2016 IEEE Congress on Evolutionary Computation (CEC), Vancouver, BC, Canada, 2016: 1998-2005.
[10] Shrestha D, Tamrakar U, Malla N, et al.Reduction of energy consumption of virtual synchronous machine using supplementary adaptive dynamic programming[C]// IEEE International Conference on Electro Information Technology, Grand Forks, ND, USA, 2016: 0690-0694.
[11] 刘向昕, 曹晓冬, 谭国俊, 等. 电励磁同步电机全速域自适应模型预测控制[J]. 电工技术学报, 2017, 32(4): 112-122.
Liu Xiangxin, Cao Xiaodong, Tan Guojun, et al.Full speed adaptive model predictive control for electrical excitation synchronous motor[J]. Transactions of China Electrotechnical Society, 2017, 32(4): 112-122.
[12] 崔淑梅, 匡志, 杜博超, 等. 基于自抗扰控制原理的全电飞机用永磁同步电机转速闭环控制[J]. 电工技术学报, 2017, 32(增刊1): 107-115.
Cui Shumei, Kuang Zhi, Du Bochao, et al.Speed closed-loop control of permanent magnet synchronous motor for all-electric aircraft applications based on active disturbance rejection controller[J]. Transactions of China Electrotechnical Society, 2017, 32(S1): 107-115.
[13] 张洪帅, 王平, 韩邦成. 基于模糊PI模型参考自适应的高速永磁同步电机转子位置检测[J]. 中国电机工程学报, 2014, 34(12): 1889-1896.
Zhang Hongshuai, Wang Ping, Han Bangcheng.Rotor position measurement for high-speed permanent magnet synchronous motors based on fuzzy PI MRAS[J]. Proceedings of the CSEE, 2014, 34(12): 1889-1896.
[14] 樊英, 周晓飞, 张向阳, 等. 基于新型趋近律和混合速度控制器的IPMSM调速系统滑模变结构控制[J]. 电工技术学报, 2017, 32(5): 9-18.
Fan Ying, Zhou Xiaofei, Zhang Xiangyang, et al.Sliding mode control of IPMSM system based on a new reaching law and a hybrid speed controller[J]. Transactions of China Electrotechnical Society, 2017, 32(5): 9-18.
[15] 王爽, 冯坚栋, 丁雪, 等. 基于最优开环截止频率学习的永磁伺服系统PI控制器参数整定方法[J]. 电工技术学报, 2017, 32(21): 44-54.
Wang Shuang, Feng Jiandong, Ding Xue, et al.PI controllers tuning method of permanent magnet servo system based on optimal open-loop cut-off frequency learning[J]. Transactions of China Electrotechnical Society, 2017, 32(21): 44-54.
[16] 符慧, 左月飞, 刘闯, 等. 永磁同步电机转速环的一种变结构PI控制器[J]. 电工技术学报, 2015, 30(12): 237-242.
Fu Hui, Zuo Yuefei, Liu Chuang, et al.A variable structure PI controller for permanent magnetic synchronous motor speed-regulation system[J]. Transactions of China Electrotechnical Society, 2015, 30(12): 237-242.
[17] Xia Changliang, Xue Mei, Chen Ziran.Adaptive PID control and on-line identification for switched reluctance motors based on BP neural network[C]//IEEE International Conference Mechatronics and Automation, Niagara Falls, Ont., Canada, 2005, 4: 1918-1922.
[18] Landelius T, Knutsson H.Greedy adaptive critics for LQR problems: convergence proofs[R]. Report LiTH-ISY-R-1896. Linkoping, Sweden: Computer Vision Laboratory, 1997.
[19] Wang Weihua, Xiao Xi, Ding Youshuang.An adaptive incremental predictive current control method of PMSM[C]//2013 15th European Conference on Power Electronics and Applications, Lille, France, 2013: 1-8.