{"title": "Refining PID Controllers using Neural Networks", "book": "Advances in Neural Information Processing Systems", "page_first": 555, "page_last": 562, "abstract": "", "full_text": "Refining PIn Controllers using Neural Networks \n\nGary M. Scott \n\nJude W. Shavlik \n\nDepartment of Chemical Engineering \n\nDepartment of Computer Sciences \n\n1415 Johnson Drive \n\nUniversity of Wisconsin \n\nMadison, WI 53706 \n\n1210 W. Dayton Street \nUniversity of Wisconsin \n\nMadison, WI 53706 \n\nW. Harmon Ray \n\nDepartment of Chemical Engineering \n\n1415 Johnson Drive \n\nUniversity of Wisconsin \n\nMadison, WI 53706 \n\nAbstract \n\nThe KBANN approach uses neural networks to refine knowledge that can \nbe written in the form of simple propositional rules. We extend this idea \nfurther by presenting the MANNCON algorithm by which the mathematical \nequations governing a PID controller determine the topology and initial \nweights of a network, which is further trained using backpropagation. We \napply this method to the task of controlling the outflow and temperature \nof a water tank, producing statistically-significant gains in accuracy over \nboth a standard neural network approach and a non-learning PID con(cid:173)\ntroller. Furthermore, using the PID knowledge to initialize the weights of \nthe network produces statistically less variation in testset accuracy when \ncompared to networks initialized with small random numbers. \n\n1 \n\nINTRODUCTION \n\nResearch into the design of neural networks for process control has largely ignored \nexisting knowledge about the task at hand. One form this knowledge (often called \nthe \"domain theory\") can take is embodied in traditional controller paradigms. The \n555 \n\n\f556 \n\nScott, Shavlik, and Ray \n\nrecently-developed KBANN (Knowledge-Based Artificial Neural Networks) approach \n(Towell et al., 1990) addresses this issue for tasks for which a domain theory (written \nusing simple propositional rules) is available. The basis of this approach is to use \nthe existing knowledge to determine an appropriate network topology and initial \nweights, such that the network begins its learning process at a \"good\" starting \npoint. \n\nThis paper describes the MANNCON (Multivariable Artificial Neural Network Con(cid:173)\ntrol) algorithm, a method of using a traditional controller paradigm to determine \nthe topology and initial weights of a network . The used of a PID controller in this \nway eliminates network-design problems such as the choice of network topology \n(i.e., the number of hidden units) and reduces the sensitivity of the network to the \ninitial values of the weights. Furthermore, the initial configuration of the network \nis closer to its final state than it would normally be in a randomly-configured net(cid:173)\nwork. Thus, the MANNCON networks perform better and more consistently than \nthe standard, randomly-initialized three-layer approach. \n\nThe task we examine here is learning to control a Multiple-Input, Multiple-Output \n(MIMO) system. There are a number of reasons to investigate this task using neu(cid:173)\nral networks. One, it usually involves nonlinear input-output relationships, which \nmatches the nonlinear nature of neural networks. Two, there have been a number \nof successful applications of neural networks to this task (Bhat & McAvoy, 1990; \nJordan & Jacobs, 1990; Miller et al., 1990). Finally, there are a number of existing \ncontroller paradigms which can be used to determine the topology and the initial \nweights of the network. \n\n2 CONTROLLER NETWORKS \n\nThe MANNCON algorithm uses a Proportional-Integral-Derivative (PID) controller \n(Stephanopoulos, 1984), one of the simplest of the traditional feedback controller \nschemes, as the basis for the construction and initialization of a neural network con(cid:173)\ntroller. The basic idea of PID control is that the control action u (a vector) should \nbe proportional to the error, the integral of the error over time, and the temporal \nderivative of the error. Several tuning parameters determine the contribution of \nthese various components. Figure 1 depicts the resulting network topology based \non the PID controller paradigm. The first layer of the network, that from Y $P (de(cid:173)\nsired process output or setpoint) and Y(n-l) (actual process output of the past time \nstep), calculates the simple error (e). A simple vector difference, \n\ne=Y$p-Y \n\naccomplishes this. The second layer, that between e, e(n-l), and e, calculates the \nactual error to be passed to the PID mechanism. In effect, this layer acts as a \nsteady-state pre-compensator (Ray, 1981), where \n\ne = GIe \n\nand produces the current error and the error signals at the past two time steps. \nThis compensator is a constant matrix, G I , with values such that interactions at a \nsteady state between the various control loops are eliminated. The final layer , that \nbetween e and u(n) (controller output/plant input), calculates the controller action \n\n\fRefining PID Controllers using Neural Networks \n\n557 \n\nFd \nTd \n\nden) Water F \n\nTank \n\nT \nYen) \n\nWCO \nWHO \nWCI \nWHI \nWC2 \nWH2 \n\nY(n-I) \n\nt:(n-I) \n\nFigure 1: MANNCON network showing weights that are initialized using \n\nZiegler-Nichols tuning parameters. \n\nbased on the velocity form of the discrete PID controller: \n\nUC(n) = UC(n-l) + WCOCI(n) + WCICI(n-l) + WC2 CI(n-2) \n\nwhere Wca, wCb and WC2 are constants determined by the tuning parameters of the \ncontroller for that loop. A similar set of equations and constants (WHO, WHI, WH2) \nexist for the other controller loop. \n\nFigure 2 shows a schematic of the water tank (Ray, 1981) that the network con(cid:173)\ntrols. This figure also shows the controller variables (Fc and FH), the tank output \nvariables (F(h) and T), and the disturbance variables (Fd and Td). The controller \ncannot measure the disturbances, which represent noise in the system. \n\nMANN CON initializes the weights of Figure 1 's network with va.lues that mimic \nthe behavior of a PID controller tuned with Ziegler-Nichols (Z-N) parameters \n(Stephanopoulos, 1984) at a particular operating condition. Using the KBANN \napproach (Towell et al., 1990), it adds weights to the network such that all units \nin a layer are connected to all units in all subsequent layers, and initializes these \nweights to small random numbers several orders of magnitude smaller than the \nweights determined by the PID parameters. We scaled the inputs and outputs of \nthe network to be in the range [0,1]. \n\nInitializing the weights of the network in the manner given above assumes that the \nactivation functions of the units in the network are linear, that is, \n\n\f558 \n\nScott, Shavlik, and Ray \n\nCold Stream \n\nFe \n\nHot Stream (at TH) \n\nT = Temperature \nF = Flow Rate \n\nI-\n\nDis t urban ce \n\nFd,Td \n\nI-\n\nl-\n\n~ \n\nh \n\nI \n\nI I \n\nOutput \nF(h), T \n\nFigure 2: Stirred mixing tank requiring outflow and temperature control. \n\nTable 1: Topology and initialization of networks. \n\nNetwork \n1. Standard neural network 3-layer (14 hidden units) \n2. MANNCON network I \n3. MANNCON network II \n\nPID topology \nPID topology \n\nTopology \n\nWeight Initialization \nrandom \nrandom \nZ-N tuning \n\nThe strength of neural networks, however, lie in their having nonlinear (typically \nsigmoidal) activation functions. For this reason, the MANNCON system initially sets \nthe weights (and the biases of the units) so that the linear response dictated by the \nPID initialization is approximated by a sigmoid over the output range of the unit. \nFor units that have outputs in the range [-1,1]' the activation function becomes \n\n2 \n\n1 + exp( -2.31 L WjiOi) \n\n_ 1 \n\nwhere Wji are the linear weights described above. \n\nOnce MANNCON configures and initializes the weights of the network, it uses a set \nof training examples and backpropagation to improve the accuracy of the network. \nThe weights initialized with PID information, as well as those initialized with small \nrandom numbers, change during backpropagation training. \n\n3 EXPERIMENTAL DETAILS \n\nWe compared the performance of three networks that differed in their topology \nand/or their method of initialization. Table 1 summarizes the network topology \nIn this table, \"PID topology\" \nand weight initialization method for each network. \nis the network structure shown in Figure 1. \"Random\" weight initialization sets \n\n\fRefining PID Controllers using Neural Networks \n\n559 \n\nTable 2: Range and average duration of setpoints for experiments. \n\nExperiment Training Set Testing Set \n\n1 \n\n2 \n\n3 \n\n[0.1,0.9] \n\n[0.1,0.9] \n\n22 instances \n\n22 instances \n\n[0.1,0.9] \n\n[0.1,0.9] \n\n22 instances \n\n80 instances \n\n[0.4,0.6] \n\n[0.1,0.9] \n\n22 instances \n\n80 instances \n\nall weights to small random numbers centered around zero. We also compare these \nnetworks to a (non-learning) PID controller. \n\nWe trained the networks using backpropagation over a randomly-determined sched(cid:173)\nule of setpoint YsP and disturbance d changes that did not repeat. The setpoints, \nwhich represent the desired output values that the controller is to maintain, are the \ntemperature and outflow of the tank. The disturbances, which represent noise, are \nthe inflow rate and temperature of a disturbance stream. The magnitudes of the \nsetpoints and the disturbances formed a Gaussian distribution centered at 0.5. The \nnumber of training examples between changes in the setpoints and disturbances \nwere exponentially distributed. \n\nWe performed three experiments in which the characteristics of the training and/or \ntesting set differed. Table 2 summarizes the range of the setpoints as well as their \naverage duration for each data set in the experiments. As can be seen, in Experiment \n1, the training set and testing sets were qualitatively similar; in Experiment 2, the \ntest set was of longer duration setpoints; and in Experiment 3, the training set was \nrestricted to a subrange of the testing set. We periodically interrupted training and \ntested the network . Results are averaged over 10 runs (Scott, 1991). \n\nWe used the error at the output of the tank (y in Figure 1) to determine the network \nerror (at u) by propagating the error backward through the plant (Psaltis et al., \n1988). In this method, the error signal at the input to the tank is given by \n\n8u i = \n\nf '( \n\n) ~ \u00b0Yi \nnetui ~ 8y j OUi \n\nwhere 8yj represents the simple error at the output of the water tank and 8ui is the \nerror signal at the input of the tank . Since we used a model of the process and not a \nreal tank, we can calculate the partial derivatives from the process model equations. \n\nJ \n\n4 RESULTS \n\nFigure 3 compares the performance of the three networks for Experiment 1. As can \nbe seen, the MANNCON networks show an increase in correctness over the standard \nneural network approach. Statistical analysis of the errors using a t-test show \nthat they differ significantly at the 99.5% confidence level. Furthermore, while the \ndifference in performance between MANNCON network I and MANNCON network II is \n\n\f560 \n\nScott, Shavlik, and Ray \n\nl~---------------------------------------------, \n\n1 = Standard neural network \n2 = MANNCON network I \n3 = MANN CON network II \n4 = PID controller (non-learning) \n\n10000 \n\n15000 \n\n20000 \n\n25000 \n\n30000 \n\nTraining Instances \n\nFigure 3: Mean square error of networks on the testset as a function of \nthe number of training instances presented for Experiment 1. \n\nnot significant, the difference in the variance of the testing error over different runs \nis significant (99.5% confidence level). Finally, the MANNCON networks perform \nsignificantly better (99.95% confidence level) than the non-learning PID controller. \nThe performance of the standard neural network represents the best of several trials \nwith a varying number of hidden units ranging from 2 to 20. \n\nA second observation from Figure 3 is that the MANNCON networks learned much \nmore quickly than the standard neural-network approach. The MANNCON networks \nrequired significantly fewer training instances to reach a performance level within \n5% of its final error rate. For each of the experiments, Table 3 summarizes the \nfinal mean error, as well as the number of training instances required to achieve a \nperformance within 5% of this value. \nIn Experiments 2 and 3 we again see a significant gain in correctness of the MAN(cid:173)\nNCON networks over both the standard neural network approach (99.95% confidence \nlevel) as well as the non-learning PID controller (99.95% confidence level). In these \nexperiments, the MANNCON network initialized with Z-N tuning also learned sig(cid:173)\nnificantly quicker (99.95% confidence level) than the standard neural network. \n\n5 FUTURE WORK \n\nOne question is whether the introduction of extra hidden units into the network \nwould improve the performance by giving the network \"room\" to learn concepts \nthat are outside the given domain theory. The addition of extra hidden units as \nwell as the removal of unneeded units is an area with much ongoing research. \n\n\fRefining PID Controllers using Neural Networks \n\n561 \n\nMethod \n\nTable 3: Comparison of network performance. \n\nI Mean Square Error I Training Instances \nExperiment 1 \n\nl. Standard neural network \n2. MANN CON network I \n3. MANN CON network II \n4. PID control (Z-N tuning) 0.0109 \n0.0190 \n5. Fixed control action \n\n0.0103 \u00b1 0.0004 \n0.0090 \u00b1 0.0006 \n0.0086 \u00b1 0.0001 \n\nExperiment 2 \n\nl. Standard neural network \n2. MANN CON network I \n3. MANN CON network II \n4. PID control (Z-N tuning) 0.0045 \n5. Fixed con trol action \n0.0181 \n\n0.0118 \u00b1 0.00158 \n0.0040 \u00b1 0.00014 \n0.0038 \u00b1 0.00006 \n\nExperiment 3 \n\nl. Standard neural network \n2. MANN CON network I \n3. MANN CON network II \n4. PID control (Z-N tuning) \n5. Fixed control action \n\n0.0112 \u00b1 0.00013 \n0.0039 \u00b1 0.00008 \n0.0036 \u00b1 0.00006 \n0.0045 \n0.0181 \n\n25,200 \u00b1 2, 260 \n5,000 \u00b1 3,340 \n640\u00b1 200 \n\n14,400 \u00b1 3, 150 \n12, 000 \u00b1 3,690 \n2,080\u00b1 300 \n\n25,200 \u00b1 2, 360 \n25,000 \u00b1 1, 550 \n9,400 \u00b1 1,180 \n\nThe \"\u00b1\" indicates that the true value lies within these bounds at a 95% \nconfidence level. The values given for fixed control action (5) represent \nthe errors resulting from fixing the control actions at a level that produces \noutputs of [0.5,0.5) at steady state. \n\n\"Ringing\" (rapid changes in controller actions) occurred in some of the trained \nnetworks . A future enhancement of this approach would be to create a network \narchitecture that prevented this ringing, perhaps by limiting the changes in the \ncontroller actions to some relatively small values. \n\nAnother important goal of this approach is the application of it to other real-world \nprocesses. The water tank in this project, while illustrative of the approach , was \nquite simple. Much more difficult problems (such as those containing significant \ntime delays) exist and should be explored. \n\nThere are several other controller paradigms that could be used as a basis for net(cid:173)\nwork construction and initialization. There are several different digital controllers, \nsuch as Deadbeat or Dahlin's (Stephanopoulos, 1984), that could be used in place \nof the digital PID controller used in this project. Dynamic Matrix Control (DMC) \n(Pratt et al., 1980) and Internal Model Control (IMC) (Garcia & Morari, 1982) are \nalso candidates for consideration for this approach. \n\nFinally, neural networks are generally considered to be \"black boxes,\" in that their \ninner workings are completely uninterpretable. Since the neural networks in this \napproach are initialized with information, it may be possible to interpret the weights \nof the network and extract useful information from the trained network. \n\n\f562 \n\nScott, Shavlik, and Ray \n\n6 CONCLUSIONS \n\nWe have described the MANNCON algorithm, which uses the information from a \nPID controller to determine a relevant network topology without resorting to trial(cid:173)\nand-error methods. In addition, the algorithm, through initialization of the weights \nwith prior knowledge, gives the backpropagtion algorithm an appropriate direction \nin which to continue learning. Finally, we have shown that using the MANNCON \nalgorithm significantly improves the performance of the trained network in the fol(cid:173)\nlowing ways: \n\n\u2022 Improved mean testset accuracy \n\u2022 Less variability between runs \n\u2022 Faster rate of learning \n\u2022 Better generalization and extrapolation ability \n\nAcknowledgements \n\nThis material based upon work partially supported under a National Science Foun(cid:173)\ndation Graduate Fellowship (to Scott), Office of Naval Research Grant N00014-90-\nJ-1941, and National Science Foundation Grants IRI-9002413 and CPT-8715051. \n\nReferences \nBhat, N. & McAvoy, T. J. (1990). Use of neural nets for dynamic modeling and \ncontrol of chemical process systems. Computers and Chemical Engineering, 14, \n573-583. \n\nGarcia, C. E. & Morari, M. (1982). Internal model control: 1. A unifying review \n\nand some new results. I&EC Process Design & Development, 21, 308-323. \n\nJordan, M. I. & Jacobs, R. A. (1990). Learning to control an unstable system \nwith forward modeling. In Advances in Neural Information Processing Systems \n(Vol. 2, pp. 325- 331). San Mateo, CA: Morgan Kaufmann. \n\nMiller, W. T., Sutton, R. S., & Werbos, P. J. (Eds.)(1990). Neural networks for \n\ncontrol. Cambridge, MA : MIT Press. \n\nPratt, D. M., Ramaker, B. L., & Cutler, C. R. (1980) . Dynamic matrix control \n\nmethod. Patent 4,349,869, Shell Oil Company. \n\nPsaltis, D., Sideris, A., & Yamamura, A. A. (1988). A multilayered neural network \n\ncontroller. IEEE Control Systems Magazine, 8, 17- 21. \n\nRay, W . H. (1981). Advanced process control. New York: McGraw-Hill, Inc. \nScott, G. M. (1991). Refining PID controllers using neural networks. Master's \n\nproject, University of Wisconsin, Department of Computer Sciences. \n\nStephanopoulos, G. (1984). Chemical process control: An introduction to theory \n\nand practice. Englewood Cliffs, NJ: Prentice Hall, Inc. \n\nTowell, G., Shavlik, J., & Noordewier, M. (1990). Refinement of approximate do(cid:173)\nmain theories by knowledge-base neural networks. In Eighth National Confer(cid:173)\nence on Aritificial Intelligence (pp. 861-866). Menlo Park, CA: AAAI Press . \n\n\f", "award": [], "sourceid": 503, "authors": [{"given_name": "Gary", "family_name": "Scott", "institution": null}, {"given_name": "Jude", "family_name": "Shavlik", "institution": null}, {"given_name": "W.", "family_name": "Ray", "institution": null}]}