JP2004118658A5 - - Google Patents

Description

The ideal algorithm avoids numerical problems due to redundancy in the input data, eliminates anomalies in the input values, maintains high data efficiency while keeping the computational complexity of the update process during learning, high dimensional In order to be able to perform learning in space in real time, it is of course necessary that accurate function approximation be possible and sufficiently generalizable. Furthermore, what is particularly problematic in performing function approximation in learning control is that in many cases the operating range is unknown and can only be defined at the upper limit. When performing function approximation if the estimated increased so the operating range, the computational cost because it must assign a number of learning parameters increases. Furthermore, if these parameters are not properly restricted by the learning data, there is a risk that they will overfit against noise. In general, when the complexity of the function to be estimated is unknown, it is difficult to determine how many to choose as the number of learning parameters, especially when learning is performed online.

The parameter θ _k needs to be approximated from the data given in the form (x _i , y _i ) or (x _i , e _i ). Here, y _i is a target of learning, and e _i is an error signal that approximates the estimation error e _{p, I} = f (x _i )-^ f (x _i ), and includes noise with an average value of 0. .

K. S. Narendra and A. M. Published by Anaswami, "Stabilized Adaptive Systems", Prentice Hall, 1989 (K.S. Narendra and A. M. Annaswamy, Stable Adaptive Systems. Prentice Hall, 1989.) J. -J. E. Throtain and W. Lee, "Applied Nonlinear Control", published by Prentice Hall, 1991 (J.-J. E. Slotine and W. Li, Applied Nonlinear Control. Prentice Hall, 1991.) J. -J. E. Throtain and W. Lee, "On adaptive control of robot manipulators", International Journal of Robotics Research, Vol. 6, No. 3, pp. 49-50, 1987 (J.-J.E. Slotine and W. Li, "On the adaptive control of robot manipulators," International Journal of Robotics Research, vol. 6, no. 3, pp. 49-59, 1987.) L. L. Whitecom, A. A. Rizzi and D.S. E. Kodi Shek, "Comparison Experiment with New Adaptive Control for Robot Arms", IEEE Transactions on Robotics and Automation, Volume 9 pp. 59-70, February 1993 (L. L. Whitcomb, A. A. Rizzi, and D. E. Koditschek, "Comparison experiments with a new adaptive controller for robot arms," IEEE Transactions on Robotics and Automation, vol. 9, pp. 59-70, Feb. 1993.). A. U. Levin and K. S. Narendra, "Control of Nonlinear Motion Systems Using Neural Networks: Controllability and Stability", IEEE Transactions on Neural Networks, Volume 4, pp. 192-206, March 1993 (A. U. Levin and K. S. Narendra, "Control of nonlinear dynamical systems using neural networks: Controllability and stabilization," IEEE Transactions on Neural Networks, vol. 4, pp. 192- 206, Mar. 1993.). F. -C.I. Chen and H. K. Karil, "Adaptive Control of a Class of Nonlinear Discrete-Time Systems Using Neural Networks," IEEE Transactions on Automatic Control, vol. 791-801, May 1995 (F.-C. Chen and H. K. Khalil, "Adaptive control of a class of nonlinear discrete-time systems using neural networks," IEEE Transactionson Automatic Control, vol. 40, pp. 791-801, May 1995.) R. Sanar and J.A. -J. E. Throtain, "Gaussian Network for Direct Adaptive Control", IEEE Transactions on Neural Networks, vol. 837-863, November 1992 (R. Sanner and J.-J. E. Slotine, "Gaussian networks for direct adaptive control," IEEE Transactions on Neural Networks, vol. 3, pp. 837-863, Nov. 1992 .) S. Seshagiri and H. K. Karil, "Output feedback control of nonlinear systems using RBF neural networks", IEEE Transactions on Neural Networks, Vol. 11, pp. 69-79, January 2000 (S. Seshagiri and H. K. Khalil, “Output feedback control of nonlinear systems using RBF neural networks,” IEEE Transactions on Neural Networks, vol. 11, pp. 69-79, Jan. 2000.) J. Y. Choi and J.A. A. See Farrell, "Nonlinear Adaptive Control Using a Network of Piecewise Linear Approximations," IEEE Transactions on Neural Networks, Vol. 11, pp. 390-401, March 2000 (J. Y. Choi and J. A. Farrell, "Non-linear adaptive control using networks of piecewise linear approximations," IEEE Transactions on Neural Networks, vol. 11, pp. 390-401, Mar .2000.) C. G. Ataxone A. W. Moore, and S. Schal, "Locally Weighted Learning", Artificial Intelligence Review, Vol. 11, No. 1-5, pp. 259-324. 11-73, 1997 (CG Atkeson, A.W. Moore, and S. Schaal, “Locally weighted learning,” Artificial Intelligence Review, vol. 11, no. 1-5, pp. 11-73, 1997.) J. -J. E. Throtain and W. Lee, "Integrated Adaptive Control of Robot Manipulators", Automata, Vol. 25, No. 4, pp. 509-519, 1989 (J.-J.E. Slotine and W. Li, "Composite adaptive control of robot manipulators," Automatica, vol. 25, no. 4, pp. 509-519, 1989.) S. Vijayakumar and H.A. Ogawa, "RKH based functional analysis for accurate incremental learning," Neurocomputing, Vol. 29, No. 1-3, pp. 85-113, 1999 (S. Vijayakumar and H. Ogawa, "RKHS based functional analysis for exact incremental learning," Neurocomputing, vol. 29, no. 1-3, pp. 85-113, 1999.) S. Shall and C.I. G. Atoxon, "Constructive Incremental Learning from Local Information Only", Neural Computation, Vol. 10, No. 8, pp. 2047-2084, 1998 (S. Schaal and C. G. Atkeson, "Constructive incremental learning from only local information," Neural Computation, vol. 10, no. 8, pp. 2047-2084, 1998.) L. Lung and T.W. Sodastrom, "Theory and Practice of Recursive Identification", MIT Press, 1986 (L. Ljung and T. Soederstroem, Theory and Practice of Recursive Identification. MIT Press, 1986.) H. K. Karil, "Nonlinear Systems (2nd Edition)", Prentice Hall, 1996 (HK Khalil, Nonlinear Systems (2nd Edition). Prentice Hall, 1996.) S. Shall and C.I. G. By Atoxon, "Receptive Field Weighted Regression", Technical Report RE-H-209, ATR Human Information Processing Research Institute, 1997 (S. Schaal and CG Atkeson, "Receptive field weighted regression," Technical report RE-H-209, ATR Human Information Processing Laboratories, 1997.) H. Garbage and M. Kawato, "Neural Network Control for Closed Loop Systems Using Feedback Error Learning," Neural Networks, Vol. 933-946, 1993 (H. Gomi and M. Kawato, "Neural network control for a closed-loop system using feedback-error-learning," Neural Networks, vol. 6, pp. 933-946, 1993.)

Ｐ_kは重み付けされた入力ｘ_kに対する共分散行列の逆行列、θ_kは当該局所モデルの学習パラメータ、ｗ_kは当該局所モデルの重み、ｅはトラッキング誤差、ｅ_pkは近似誤差、λは忘却係数、
にしたがって当該局所モデルの学習パラメータの近似＾θ_kを算出するステップと、所定の式により定められる、学習データを表わす関数値ｙと関数近似＾ｙとの間で定められる誤差指標を最小化することにより、距離メトリックの各々を最適化するステップとを含んでもよい。

P _k is the inverse matrix of the covariance matrix for the input x _k weighted, theta _k learning parameters of the local models, w _k is the weight of the local model, e is the tracking error, e _pk approximation error, lambda forgetting coefficient,
Calculating an approximation ^ θ _k of the learning parameter of the local model according to and minimizing an error index defined between the function value y representing the learning data and the function approximation ^ y determined by a predetermined equation And D. optimizing each of the distance metrics.

A controller of a physical system according to still another aspect of the present invention approximates the non-linear function describing the dynamics of the physical system by a function approximation obtained by weighting and adding a linear local model. It is a control device of a physical system that controls the physical system. The structure of the local model constituting the function approximation and the respective weights are determined by predetermined learning parameters. This apparatus comprises: initialization means for defining an initial structure of function approximation; receiving means for receiving state data representing an actual state of the physical system; and a target trajectory of the physical system based on the state data Based on the tracking error between the actual trajectory and the approximation error between the state data and the function approximation, learning parameters of each local model are minimized so as to minimize a predetermined error index independently for each local model. Updating means for updating function approximation by updating, calculating means for calculating control variables according to the control law of the control system using the updated function approximation, and the calculated control variables as a physical system And output means for outputting data, and control means for controlling the receiving means, the updating means, the calculating means, and the output means to operate repeatedly.

Ｐ_kは重み付けされた入力ｘ_kに対する共分散行列の逆行列、θ_kは当該局所モデルの学習パラメータ、ｗ_kは当該局所モデルの重み、ｅはトラッキング誤差、ｅ_pkは近似誤差、λは忘却係数、
にしたがって当該局所モデルの学習パラメータの近似＾θ_kを算出するための手段と、所定の式により定められる、学習データを表わす関数値ｙと関数近似＾ｙとの間で定められる誤差指標を最小化することにより、距離メトリックの各々を最適化するための最適化手段とを含む。

P _k is the inverse matrix of the covariance matrix for the input x _k weighted, theta _k learning parameters of the local models, w _k is the weight of the local model, e is the tracking error, e _pk approximation error, lambda forgetting coefficient,
Means for calculating the approximation ^ θ _k of the learning parameter of the local model according to and the error index defined between the function value y representing the learning data and the function approximation ^ y determined by a predetermined equation And optimizing means for optimizing each of the distance metrics.

FIG. 5 is a block diagram of the controller 60 that performs non-linear control according to the present embodiment, and a sensor group 62A-62N for inputting control variables from a physical system such as a robot to be controlled to the controller 60. And the actuator group 64A-64M controlled and operated by the controller 60. The controller 60 has a CPU (Central Processing Unit) connected to the input port 70 receiving the input from the sensor group 62A to 62N, the output port 72 to which the actuator group 64A to 64M is connected, and the input port 70 and the output port 72 74 includes a ROM (Read-Only Memory) 76, a RAM (Random Access Memory) 78, a network board 82, and a memory reader 80, all of which are connected to the CPU 74. The network board 82 is connected to an external network 92. A memory card 90 made of an integrated circuit is removable from the memory reader 80. The data and program stored in the memory card 90 can be supplied to the CPU 74, and data from the CPU 74 can be stored.

Ｐ_kは重み付けされた入力ｘ_kに対する共分散行列の逆行列であり、θ_kは学習パラメータであり、ｗ_kは前述の重みであり、ｅはトラッキング誤差であり、ｅ_pkは近似誤差であり、λは忘却係数である。忘却係数λは、パラメータ更新においてはある程度新しいデータだけを用いるために導入された係数で［０，１］の値をとる。

P _k is the inverse matrix of the covariance matrix for the input x _k weighted, theta _k is the learning parameter, w _k is the weight of the above, e is the tracking error, e _pk is an approximation error , Λ are oblivion factors. The forgetting factor λ takes a value of [0, 1] with a factor introduced to use only new data to a certain extent in parameter updating.

FIG. 9 shows a flowchart of a program for adding a local model as an example of steps 122 and 124 of FIG. Referring to FIG. 9, whether initially less than all of the weights w _k is the threshold calculated for the data points x is the step 170 is determined. If the determination result is YES, it means that the presence of this data point is not sufficiently reflected in any local model, so a new local model is added in step 172. The initial value of the center c _k of the local model in this case is set to x . An appropriate initial value is set to the width. For example, the width of an adjacent local model may be set to an initial value. This is based on the assumption that adjacent local models correspond to adjacent parts of the true function, so the curvature of the true function there will not be much different. However, since the local model to be newly added here is adjusted in the subsequent update processing, it is not essential to select the width as described above. However, selecting as above has an effect that the width of the local model is adjusted to the optimum value at an early stage.

Referring again to FIG. 6 , an example of a control law used in the calculations at step 106 is of the form:

FIG. 12 shows tracking error 230 by the non-adaptive PD controller when N (0, 0.01) Gaussian noise is added to the measured value, and tracking by the tracking error based adaptive controller of Γ _k = 10 I and 250 I The errors 232 and 234 and the tracking error 236 by the RFWR complex adaptive controller of the present embodiment are compared and shown. As shown in FIG. 12, the performance of the tracking error based adaptive controller at Γ k = 250 I is greatly degraded by the presence of Gaussian noise. On the other hand, it can be seen that the RFWR complex adaptive controller according to the present embodiment realizes stable and fast learning.

[Description of the code]
20 target trajectory, 22 actual trajectory, 24 tracking error, 26, 40 true function, 28 function approximation, 30 approximation error, 32 kernel function, 42, 44, 46 kernel diameter range, 42C training point , 52, 54, 56 local approximation linear function, 60 controller, 74 CPU, 76 ROM, 78 RAM, 80 memory reader, 82 network board, 90 memory card, 92 network

Claims (20)

A control method of a physical system for controlling a physical system by approximating a non-linear function describing the dynamics of the physical system by a function approximation obtained by weighting and adding a linear local model. The structure of the local model making up the approximation and the respective weights are determined by predetermined learning parameters,
Defining an initial structure of the function approximation;
Receiving state data representing an actual state of the physical system;
Based on the state data, it is determined independently for each local model based on the tracking error between the target trajectory of the physical system and the actual trajectory, and the approximation error between the state data and the function approximation. Updating the function approximation by updating the learning parameters of each local model so as to minimize the error index of
Calculating the control variable according to the control law of the control system using the updated function approximation;
Outputting the calculated control variable to the physical system;
A control method of a physical system, comprising the steps of receiving, updating, performing calculation, and repeatedly performing outputting. The function approximation ^ y is

However

c _k is the center position of the k th linear model,
w _k is a weight represented by a predetermined kernel function,
The control method of the physical system according to claim 1, represented by The weight w _k is

The control method of a physical system according to claim 2, wherein the control method is calculated by a kernel function. The updating step is
For each of the existing local models, based on the state data, based on tracking errors between the target trajectory of the physical system and the actual trajectory, and an approximation error between the state data and the function approximation A second step of updating the learning parameter to minimize a predetermined error indicator;
Determining whether a learning parameter of each local model updated in the second step of updating satisfies a predetermined condition;
The method according to claim 3, further comprising the steps of: adding or deleting a local model in response to determining that the learning parameter of each local model satisfies the predetermined condition in the determining step. System control method. The second step of updating is
For each of the local models,
Calculating the weights w _k based on said status data and said tracking error,
Using the weight w _k , the following equation

P _k is the inverse matrix of the covariance matrix for the input x _k weighted, theta _k learning parameters of the local models, w _k is the weight of the local model, e is the tracking error, e _pk approximation error, lambda forgetting Calculating an approximation ^ θ _k of the learning parameter of the local model according to the coefficient;
Optimizing each of the distance metrics by minimizing an error indicator defined between the function value y representing the training data and the function approximation y y defined by the predetermined equation. The control method of the physical system as described in. The determining step includes the step of determining whether the weights w _k (k = 1 to the number of local models) calculated for all the local models are less than a predetermined threshold value.
Wherein the step of adding or deleting includes the step of all the weights w _k calculated for the local model in response to the determination that below a predetermined threshold, adding a new local model, wherein Item 5. A control method of a physical system according to item 5.   The control method of a physical system according to claim 6, wherein an initial value of a center position of the local model added in the adding step is selected to be equal to a data point corresponding to the state data. The control method of a physical system according to claim 6 or 7, wherein the initial value of the width of the local model added in the adding step is selected to be equal to the width of the local model closest to the added local model. . The optimizing step includes the step of optimizing the distance metric D _{k, ij} so as to minimize the error indicator J _k defined by the following equation:

Here, the following gradient descent method is used,

here,

The control method of the physical system according to any one of claims 6 to 8, wherein γ is a scalar quantity that determines the size of the penalty, and α is a learning rate.   A computer program for control of a physical system, comprising computer program code means configured to execute the control method of a physical system according to any one of claims 1 to 9 by being executed on a computer. .   A computer program for controlling a physical system according to claim 10, recorded on a computer readable storage medium. A control system for a physical system, which controls a physical system by approximating a non-linear function describing the dynamics of the physical system by a function approximation obtained by weighting and adding a linear local model. The structure of the local model making up the approximation and the respective weights are determined by predetermined learning parameters,
Initialization means for defining an initial structure of the function approximation;
Receiving means for receiving state data representing an actual state of the physical system;
Based on the state data, it is determined independently for each local model based on the tracking error between the target trajectory of the physical system and the actual trajectory, and the approximation error between the state data and the function approximation. Updating means for updating the function approximation by updating learning parameters of each local model so as to minimize an error index of
Calculation means for calculating control variables according to the control law of the control system using the updated function approximation;
Output means for outputting the calculated control variable to the physical system;
A control system for a physical system, comprising: the receiving means, the updating means, the calculating means, and a control means for controlling the output means to operate repeatedly. The function approximation ^ y is

However

c _k is the center position of the k th linear model,
w _k is a weight represented by a predetermined kernel function,
The control device of a physical system according to claim 12, represented by The weight w _k is

The controller of a physical system according to claim 13, which is calculated by a kernel function The updating means is
For each of the existing local models, based on the state data, based on tracking errors between the target trajectory of the physical system and the actual trajectory, and an approximation error between the state data and the function approximation Second updating means for updating the learning parameters to minimize a predetermined error indicator;
Determining means for determining whether the learning parameter of each local model updated by the second updating means satisfies a predetermined condition;
And means for adding or deleting a local model in response to the determination means determining that the learning parameter of each local model satisfies the predetermined condition. Control system. The second updating means is
For each of the local models,
Means for calculating the weights w _k based on said status data and said tracking error,
Using the weight w _k , the following equation

P _k is the inverse matrix of the covariance matrix for the input x _k weighted, theta _k learning parameters of the local models, w _k is the weight of the local model, e is the tracking error, e _pk approximation error, lambda forgetting Means for calculating an approximation ^ θ _k of the learning parameter of the local model according to the coefficient;
And optimization means for optimizing each of the distance metrics by minimizing an error indicator defined between a function value y representing the training data and a function approximation y determined by a predetermined equation The control device of a physical system according to claim 15. The determining means includes means for determining whether the weights w _k (k = 1 to the number of local models) calculated for all the local models are less than a predetermined threshold value,
Said means for adding or deleting, all weights w _k calculated for the local model in response to the determination that below a predetermined threshold, additional to add a new local model The control device of the physical system according to claim 16 including means.   The control device of a physical system according to claim 17, wherein an initial value of a central position of the local model added by the addition means is selected to be equal to a data point corresponding to the state data.   The control device of a physical system according to claim 17 or 18, wherein the initial value of the width of the local model added by the addition means is selected to be equal to the width of the local model closest to the added local model. The optimization means includes means for optimizing the distance metric D _{k, ij} to minimize an error indicator J _k defined by the following equation:

Here, the following gradient descent method is used,

here,

20. The control system of the physical system according to any one of claims 17 to 19, wherein γ is a scalar quantity that determines the size of the penalty, and α is a learning rate.