{"title": "Neural Control for Nonlinear Dynamic Systems", "book": "Advances in Neural Information Processing Systems", "page_first": 1010, "page_last": 1016, "abstract": null, "full_text": "Neural Control for Nonlinear Dynamic Systems \n\nSsu-Hsin Yu \n\nDepartment of Mechanical Engineering \nMassachusetts Institute of Technology \n\nCambridge, MA 02139 \nEmail: hsin@mit.edu \n\nAnuradha M. Annaswamy \n\nDepartment of Mechanical Engineering \nMassachusetts Institute of Technology \n\nCambridge, MA 02139 \nEmail: aanna@mit.edu \n\nAbstract \n\nA neural network based approach is presented for controlling two distinct \ntypes of nonlinear systems. The first corresponds to nonlinear systems \nwith parametric uncertainties where the parameters occur nonlinearly. \nThe second corresponds to systems for which stabilizing control struc(cid:173)\ntures cannot be determined. The proposed neural controllers are shown \nto result in closed-loop system stability under certain conditions. \n\n1 \n\nINTRODUCTION \n\nThe problem that we address here is the control of general nonlinear dynamic systems \nin the presence of uncertainties. Suppose the nonlinear dynamic system is described as \nx= f(x , u , 0) , y = h(x, u, 0) where u denotes an external input, y is the output, x is the \nstate, and 0 is the parameter which represents constant quantities in the system. The control \nobjectives are to stabilize the system in the presence of disturbances and to ensure that \nreference trajectories can be tracked accurately, with minimum delay. While uncertainties \ncan be classified in many different ways, we focus here on two scenarios. One occurs \nbecause the changes in the environment and operating conditions introduce uncertainties \nin the system parameter O. As a result, control objectives such as regulation and tracking, \nwhich may be realizable using a continuous function u = J'(x, 0) cannot be achieved since \no is unknown. Another class of problems arises due to the complexity of the nonlinear \nfunction f. Even if 0, f and h can be precisely determined, the selection of an appropriate \nJ' that leads to stabilization and tracking cannot be made in general. In this paper, we \npresent two methods based on neural networks which are shown to be applicable to both \nthe above classes of problems. In both cases, we clearly outline the assumptions made, \nthe requirements for adequate training of the neural network, and the class of engineering \nproblems where the proposed methods are applicable. The proposed approach significantly \nexpands the scope of neural controllers in relation to those proposed in (Narendra and \nParthasarathy, 1990; Levin and Narendra, 1993; Sanner and Slotine, 1992; Jordan and \nRumelhart, 1992). \n\n\fNeural Control for Nonlinear Dynamic Systems \n\n1011 \n\nThe first class of problems we shall consider includes nonlinear systems with parametric \nuncertainties. The field of adaptive control has addressed such a problem, and over the \npast thirty years, many results have been derived pertaining to the control of both linear \nand nonlinear dynamic systems (Narendra and Annaswamy, 1989). A common assumption \nin almost all of the published work in this field is that the uncertain parameters occur \nlinearly. In this paper, we consider the control of nonlinear dynamic systems with nonlinear \nparametrizations. We design a neural network based controller that adapts to the parameter \no and show that closed-loop system stability can be achieved under certain conditions. Such \na controller will be referred to as a O-adaptive neural controller. Pertinent results to this \nclass are discussed in section 2. \n\nThe second class of problems includes nonlinear systems, which despite being completely \nknown, cannot be stabilized by conventional analytical techniques. The obvious method for \nstabilizing nonlinear systems is to resort to linearization and use linear control design meth(cid:173)\nods. This limits the scope of operation of the stabilizing controller. Feedback linearization \nis another method by which nonlinear systems can be stably controlled (lsidori, 1989). This \nhowever requires fairly stringent set of conditions to be satisfied by the functions! and h. \nEven after these conditions are satisfied, one cannot always find a closed-form solution to \nstabilize the system since it is equivalent to solving a set of partial differential equations. \nWe consider in this paper, nonlinear systems, where system models as well as parameters \nare known, but controlIer structures are unknown. A neural network based controller is \nshown to exist and trained so that a stable closed-loop system is achieved. We denote this \nclass of controllers as a stable neural controller. Pertinent results to this class are discussed \nin section 3. \n\n2 O-ADAPTIVE NEURAL CONTROLLER \n\nThe focus of the nonlinear adaptive controller to be developed in this paper is on dynamic \nsystems that can be written in the d-step ahead predictor form as follows: \n\nYt+d = !r(Wt,Ut,O) \n\n(I) \n. ,Yt-n+l, Ut-I, ' \", Ut-m-d+l], n ~ I, m ~ 0, d ~ I, m + d = n, \nwhere wi = [Yt,\" \nYI, U I C ~ containing the origin and 8 1 C ~k are open, ir : Y1 x U;n+d x 8 1 - t ~, Yt \nand Ut are the output and the input of the system at time t respectively, and 0 is an unknown \nparameter and occurs nonlinearly in ir.1 The goal is to choose a control input 'It such that \nthe system in (1) is stabilized and the plant output is regulated around zero. \n\nLetxi ~ [Yt+d-I , '\" \n,Yt+l , wil T , Am = [e2,\"', en+d-I, 0 , en+d+I,\"', en +m+2d-2, \n0], Bm = [el' en+d], where e, is an unit vector with the i-th term equal to I. The following \nassumptions are made regarding the system in Eq. (I ). \n\n(AI) For every 0 E 8 1, ir(O,O, O) = 0. \nCA2) There exist open and convex neighborhoods of the origin Y2 C YI and U2 C U I, an \nopen and convex set 82 C 8 1, and a function K : 0.2 x Y2 x 8 2 ---> U I such that for \nevery Wt E 0.2, Yt+d E Y2 and 0 E 8 2, Eq. (1) can be written as Ut = K(wt, Yt+d, 0), \nwhere 0.2 ~ Y2 X u;,,+d-I. \n\n(A3) K is twice differentiable and has bounded first and second derivatives on EI ~ 0.2 X \nY2 X 8 2 , while ir is differentiable and has a bounded derivative on 0.2 x I{ (E I ) x 8 2 . \n\n(A4) There exists bg > \u00b0 such that for every YI E ir(o.2, K(o.2' 0 , 8 2), 8 2), W E 0.2 and \n\no BE 8 \n,2 , \n\nay \n\n11 - (8K(w,y ,O) _ 8K(w,y,9))1 _ \n\nay \n\nY - YI \n\n. 8f,(w ,u ,O) I -\n\nau \n\nU-UI \n\nI > b \n\ng' \n\n1 Here, as well as in the following sections, An denotes the n-th product space of the set A . \n\n\f1012 \n\nS. YU, A. M. ANNASW AMY \n\n(A5) There exist positive definite matrices P and Q of dimensions (n + m + 2d - 2) such \nthat x t (AmPAm - P)Xt+ J( BrnPBmK + 2xt ArnPBmK ~ -Xt QXt, where \n[( = [0, K(wt, 0 , O)]T. \n\n-T T \n\nT T' \n\nT T \n\nT \n\n-\n\n-\n\nSince the objective is to control the system ~n (1) where 0 is unknown, in order to stabilize \nthe output Y at the origin with an estimate Of, we choose the control input as \n\n(2) \n\n2.1 PARAMETER ESTIMATION SCHEME \n\nSuppose the estimation algorithm for updating Ot is defined recursively as /10t ~ Ot(cid:173)\nOt-I = R(Yt,Wt-d,Ut-d,Ot-l) the problem is to determine the function R such that Ot \nconverges to 0 asymptotical1y. In general, R is chosen to depend on Yt, Wt-d, 1\u00a3t-d and \nOt-l since they are measurable and contain information regarding O. For example, in the \ncase of linear systems which can be cast in the input predictor form, 1\u00a3t = **'t~~t-'/ [1\u00a3t-d - \u00a2LdOt-d\u00b7 In other words, the mechanism for carrying out \nparameter estimation is realized by R. In the case of general nonlinear systems, the task \nof determining such a function R is quite difficult, especial\\y when the parameters occur \nnonlinearly. Hence, we propose the use of a neural network parameter estimation algorithm \ndenoted O-adaptive neural network (TANN) (Annaswamy and Yu, 1996). That is, we adjust \nOt as \n\nif /1Vd, < - f \notherwise \n\n(3) \n\nwhere the inputs of the neural network are Yt, Wt-d, 1\u00a3t-d and Ot-I, the output is /10t, and \nf defines a dead-zone where parameter adaptation stops. \n\nThe neural network is to be trained so that the resulting network can improve the pa(cid:173)\nrameter estimation over time for any possible 0 in a compact set. \nIn addition, the \ntrained network must ensure that the overal1 system in Eqs. (1), (2) and (3) is stable. \nToward this end, N in TANN algorithm is required to satisfy the fol1owing two proper-\nties: (PI) IN(Yt,wt - d,1\u00a3t - d,Ot-l)12 ~ a(llfJt;~~,~1~2)2uLd' and (P2) /1Vt -/1Vd, < fl, \nt- d' \nC(\u00a2t) = (~~ (Wt,Yt+d,O)lo=oo)T, Ut = Ut - K(Wt,Yt+d,Ot+d - I), (fit = [WT,Yt+djT, \na E (0, I) and 00 is the point where K is linearized and often chosen to be the mean value \nof parameter variation. \n\n-a 2+ IC( 0 where A T f \n\nIii 12 _ Iii \nUt \n\nUt - I, Ut - Ut \n\nII _ II AV, \n\nu, L.l. d, -\n\n, 4>,-./ \n\nL.l.Vt -\n\n-\n\nIC( -\n\n(\n\n1+ \n\n12 \n\nii \n\n-\n\nfl \n\n, \n\n-\n\n1\u00a3-2 \n\n2.2 TRAINING OF TANN FOR CONTROL \n\nIn the previous section, we proposed an algorithm using a neural network for adjusting \nthe control parameters. We introduced two properties (PI) and (P2) of the identification \nalgorithm that the neural network needs to possess in order to maintain stability of the \nclosed-loop system. In this section, we discuss the training procedure by which the weights \nof the neural network are to be adjusted so that the network retains these properties, \n\nThe training set is constructed off-line and should compose of data needed in the train(cid:173)\ning phase. If we wan..!. the algorithm in Eq. (3) to be valid on the specified sets Y3 and \nU3 for various 0 and 0 in 83, the training set should cover those variables appearing in \nEq. (3) in their respective ranges. Hence, we first sample W in the set Y;- x U;:+d-I, \n\n\fNeural Control for Nonlinear Dynamic Systems \n\n1013 \n\nth \n\necorrespon mg \n\nd\u00b7 C(A-) - BK ( \n\nand B, 8 in the set 83. Their values are, say, WI, BI and 81 respectively. For the \nparticular fh and BI we sample 8 again in the set {B E 8 31 IB - BII \n:s: 181 - BI I}, \nand its value is, say, 8t. Once WI, BI , 81 and 8t are sampled, other data can then \nbe calculated, such as UI = K(WI' 0, 8d and YI = fr(WI, UI, Bd. We can also ob-\n~)2 d \n. \ntam \nan \n_ \n\n_ \nLI - a (1+IC(***I)I2)2 (UI - UI) ,where \u00a2I -\nand UI - K(WI' YI,( 1 )\u00b7 A data \nelement can then be formed as (Yl ,WI ,UI, 8t, BI , ~ Vd l , Ld. Proceeding in the same man(cid:173)\nner, by choosing various ws , Bs , 1f. and 8~ in their respective ranges, we form a typical \ntraining set Ttram = { (Ys , W s, us,1f~ , Bs, ~ Vd d Ls) 11 :s: s :s: M}, where M denotes the \ntotal number of patterns in the training set. If the quadratic penalty function method (Bert(cid:173)\nsekas, 1995) is used, properties (PI) and (P2) can be satisfied by training the network on \nthe training set to minimize the following cost function: \n\n- ao WI , YI, 0, i l d l \nB) All: \nT \n[WI ,yJ} \n\n-a(I+IC(\u00a2I)i2)2 UI -'ttl \n\n2+IC(\u00a2dI 2 ( \n\n'1'1 \n~ 2 \n\nIC(\u00a2IW \n\n~ _ \n\n~d \n\n-\n\n-\n\n-\n\nT \n\nmJpl ~ mJ,n~~{(max{0, ~VeJ)2+ ;2 (max{0, INi(W)12 - L t})2} \n\nM \n\n(4) \n\nTo find a W which minimizes the above unconstrained cost function 1, we can apply \nalgorithms such as the gradient method and the Gauss-Newton method. \n\n2.3 STABILITY RESULT \n\nWith the plant given by Eq. (1), the controller by Eq. (2), and the TANN parameter \nestimation algorithm by Eq. (3), it can be shown that the stability of the closed-loop system \nis guaranteed. \n\nBased on the assumptions of the system in (1) and properties (PI) and (P2) that TANN \nsatisfies, the stability result of the closed-loop system can be concluded in the following \ntheorem. We refer the reader to (Yu and Annaswamy, 1996) for further detail. \n\nTheorem 1 Given the compact sets Y;+ I X U:;+d x 8 3 where the neural network in Eq. (3) \nis trained. There exist EI, E > 0 such that for any interior point B of 8 3, there exist open \nsets Y4 C Y3, U4 C U3 and a neighborhood 8 4 of B such that if Yo , ... , Yn+d-2 E Y4, \nl , ... ,8n +d - 2 E 8 4, then all the signals in the closed-loop \nUo, .. . , U n -2 E U4 , and 8n -\nsystem remain bounded and Yt converges to a neighborhood of the origin. \n\n2.4 SIMULATION RESULTS \n\n. \n\n. \n\nTh \n\ne system IS 0 \n\nf h f \nt e orm Yt+1 = I+e U.USH\", + Ut, were \n\nIn this section, we present a simulation example of the TANN controller proposed in this \nb \nsectton. \nIS t e parameter to e \ndetermined on-line. Prior information regarding the system is that () E [4, 10]. Based on \n' where Bt denotes the parameter \nEq. (2), the controller was chosen to be Ut = -\nestimate at time t. According to Eq. (3), B was estimated using the TANN algorithm with \ninputs YHI, Yt. Ut and~, and E = 0.01. N is a Gaussian network with 700 centers. The \ntraining set and the testing set were composed of 6,040 and 720 data elements respectively. \n\n8 ( I ) \n,y, 0 ,-;'Y' \nI+e - \u00b7 \"\" \n\nB\u00b7 h \n\nlIy, ( I-y,) \n\n~ \n\nh \n\nAfter the training was completed, we tested the TANN controller on the system with six \ndifferent values of B, 4.5, 5.5, 6.5, 7.~, 8.5 and 9.5, while the initial parameter estimate \nand the initial output were chosen as BI = 7 and Yo = -0.9 respectively. The results are \nplotted in Figure 1. It can be seen that Yt can be stabilized at the origin for all these values \nof B. For comparison, we also simulated the system under the same conditions but with 8 \n\n\f1014 \n\n-1 \n\n-2 \no \n\nS. YU, A. M. ANNASWAMY \n\n~ 0 \n\n-1 \n\n-2 \no \n\n50 \n\n100 4 \n\n10 \n\n50 \n\n10 \n\n100 4 \n\nFigure 1: Yt (TANN Controller) \n\nFigure 2: Yt (Extended Kalman Filter) \n\nestimated using the extended Kalman filter (Goodwin and Sin, 1984). Figure 2 shows the \noutput responses. It is not surprising that for some values of fJ, especially when the initial \nestimation error is large, the responses either diverge or exhibit steady state error. \n\n3 STABLE NEURAL CONTROLLER \n\n3.1 STATEMENT OF THE PROBLEM \n\nConsider the following nonlinear dynamical system \n\nX= j(x,u), \n\nY = h(x) \n\n(5) \n\nwhere x E Rn and u E RTn. Our goal is to construct a stabilizing neural controller as \nu = N(y; W) where N is a neural network with weights W, and establish the conditions \nunder which the closed-loop system is stable. \n\nThe nonlinear system in (5) is expressed as a combination of a higher-order linear part and \na nonlinear part as x= A x + Bu + RI (x, u) and y = Cx + R 2(x), where j(O, O) = 0 \nand h(O) = O. We make the following assumptions: (AI) j, h are twice continuously \ndifferentiable and are completely known. (A2) There exists a K such that (A - BKC) is \nasymptotically stable. \n\n3.2 TRAINING OF THE STABLE NEURAL CONTROLLER \n\nIn order for the neural controller in Section 3.1 to result in an asymptotically stable c1osed(cid:173)\nloop system, it is sufficient to establish that a continuous positive definite function of the state \nvariables decreases monotonically through output feedback. In other words, if we can find a \nscalar definite function with a negative definite derivative of all points in the state space, we \ncan guarantee stability of the overall system. Here, we limit our choices of the Lyapunov \nfunction candidates to the quadratic form, i.e. V = x T Px, where P is positive definite, \nand the goal is to choose the controller so that V < 0 where V = 2xT P j(x, N(h(x), W)). \nBased on the above idea, we define a \"desired\" time-derivative V d as V d= -xTQx where \nQ = QT > O. We choose P and Q matrices as follows. First, according to (AI), we can \nfind a matrix K to make (A - BKC) asymptotically stable. We can then find a (P, Q) \npair by choosing an arbitrary positive definite matrix Q and solving the Lyapunov equation, \n(A - BKC)T P + P(A - BKC) = -Q to obtain a positive definite P. \n\n\fNeural Control for Nonlinear Dynamic Systems \n\n1015 \n\nWith the contro\\1er of the form in Section 3.1, the goal is to find W in the neural network \nwhich yields V:::; V d along the trajectories in a neighborhood X C ~n of the origin in the \nstate space. Let Xi denote the value of a sample point where i is an index to the sample \nvariable X E X in the state space. To establish V:::; V d, it is necessary that for every Xl in \na neighborhood X C ~n of the origin, Vi:::;Vd\" where Vi= 2x;Pf(x l ,N(h(:rl\n, W)) \nand V d, = -x; QXi . That is, the goal is to find a W such that the inequality constraints \ntlVe , \n- V d, and M denotes the \ntotal number of sample points in X. As in the training of TANN controller, this can also \nbe posed as an optimization problem. If the same quadratic penalty function method is \nused, the problem is to find W to minimize the fo\\1owing cost function over the training \nset, which is described as Ttrain = {(Xl' Yi, V d.)\\l :::; i :::; M}: \n\n:::; 0, where i = 1,\u00b7\u00b7\u00b7 , M, is satisfied, where tlVe , =V l \n\n)\n\nrwn J 6. mJp 2 I: (max{O, tlVe ,})2 \n\n1M \n\ni= 1 \n\n(6) \n\n3.3 STABILITY OF THE CLOSED-LOOP SYSTEM \n\nAssum~tions (A 1) and (A2) imply that a stabilizing controller u = - J( y exists so that \nV = X Px is a candidate Lyapunov function . More genera\\1y, suppose a continuous but \nunknown function ,,((y) exists such that for V = x T Px, a control input 1t = \"((y) leads to \nV:::; -xT Qx, then we can find a neural network N (y) which approximates \"((y) arbitrarily \nclosely in a compact set leading to closed-loop stability. This is summarized in Theorem 2 \n(Yu and Annaswamy, 1995). \n\nTheorem 2 Let there be a continuous function \"((h(x)) such that 2xT P f(x , \"((h(x))) + \nxT Qx :::; 0 for every X E X where X is a compact set containing the origin as an interior \npoint. Then, given a neighborhood 0 C X of the origin, there exists a neural controlierH = \nN(h(x); W) and a compact set Y E X such that the solutions of x= f(x , N(h(x); W)) \nconverge to 0, for every initial condition x(to) E y. \n\n3.4 SIMULATION RESULTS \n\nIn this section, we show simulation results for a discrete-time nonlinear systems using the \nproposed neural network contro\\1er in Section 3, and compare it with a linear contro\\1er to \nillustrate the difference. The system we considered is a second-order nonlinear system Xt = \nf(xt-I , Ut-I), where f = [II, 12]T, h = Xl t _ 1 X (1 +X2'_ 1 )+X2t-1 x (l-u t- I +uLI) and \n12 = XT'_I + 2X2'_1 +Ut-I (1 + X2'_I)\u00b7 It was assumed that X is measurable, and we wished \nto stabilize the system around the origin. The controller is of the form Ht = N (x It, X2 t ). \nThe neural network N used is a Gaussian network with 120 centers. The training set and \nthe testing set were composed of 441 and 121 data elements respectively. \n\nAfter the training was done, we plotted the actual change of the Lyapunov function, tl V, \nusing the linear controller U = - K x and the neural network controller in Figures 3 and 4 \nrespectively. It can be observed from the two figures that if the neural network contro\\1er is \nused, tl V is negative definite except in a small neighborhood of the origin, which assures \nthat the closed-loop system would converge to vicinity of the origin; whereas, if the linear \ncontroller is used, tl V becomes positive in some region away from the origin, which implies \nthat the system can be unstable for some initial conditions. Simulation results confirmed \nour observation. \n\n\f1016 \n\nS. YU, A. M. ANNASW AMY \n\n-0 01 \n\n- 0 J \n\n-0 J \n\n- 0 I \n\n-()2 \n\n-O J \n\nFigure 3: ~V(u = -Kx ) \n\nFigure 4: ~V(u = N(x)) \n\nAcknowledgments \n\nThis work is supported in part by Electrical Power Research Institute under contract No. \n8060-13 and in part by National Science Foundation under grant No. ECS-9296070. \n\nReferences \n\n[1] A. M. Annaswamy and S. Yu. O-adaptive neural networks: A new approach to \n\nparameter estimation. IEEE Transactions on Neural Networks, (to appear) 1996. \n\n[2] D. P. Bertsekas. Nonlinear Programming. Athena Scientific, Belmont, MA, 1995. \n[3] G. C. Goodwin and K. S. Sin. Adaptive Filtering Prediction and Control. Prentice(cid:173)\n\nHall, Inc., 1984. \n\n[4] A. Isidori. Nonlinear Control Systems. Springer-Verlag, New York, NY, 1989. \n[5] M. L Jordan and D. E. Rumelhart. Forward models: Supervised learning with a distal \n\nteacher. Cognitive Science, 16:307-354, 1992. \n\n[6] A. U. Levin and K. S. Narendra. Control of nonlinear dynamical systems using neural \nnetworks: Controllability and stabilization. IEEE Transactions on Neural Networks, \n4(2): 192-206, March 1993. \n\n[7] K. S. Narendra and A . M. Annaswamy. Stable Adaptive Systems. Prentice-Hall, Inc., \n\n1989. \n\n[8] K. S. Narendra and K. Parthasarathy. Identification and control of dynamical systems \nusing neural networks. IEEE Transactions on Neural Networks, 1 (I ):4-26, March \n1990. \n\n[9] R. M. Sanner and J.-J. E. Slotine. Gaussian networks for direct adaptive control. IEEE \n\nTransactions on Neural Networks, 3(6):837-863, November 1992. \n\n[10] S. Yu and A. M. Annaswamy. Adaptive control of nonlinear dynamic systems using \nO-adaptive neural networks. Technical Report 9601 , Adaptive Control Laboratory, \nDepartment of Mechanical Engineering, M.LT., 1996. \n\n[11] S.-H. Yu and A. M. Annaswamy. Control of nonlinear dynamic systems using a \nstability based neural network approach. In Technical report 9501, Adaptive Control \nLaboratory, MIT, Submitted to Proceedings of the 34th IEEE Conference on Decision \nand Control, New Orleans, LA, 1995. \n\n\f", "award": [], "sourceid": 1043, "authors": [{"given_name": "Ssu-Hsin", "family_name": "Yu", "institution": null}, {"given_name": "Anuradha", "family_name": "Annaswamy", "institution": null}]}*