Part of Advances in Neural Information Processing Systems 3 (NIPS 1990)
N. Toomarian, J. Barhen
The development of learning algorithms is generally based upon the min(cid:173) imization of an energy function. It is a fundamental requirement to com(cid:173) pute the gradient of this energy function with respect to the various pa(cid:173) rameters of the neural architecture, e.g., synaptic weights, neural gain,etc. In principle, this requires solving a system of nonlinear equations for each parameter of the model, which is computationally very expensive. A new methodology for neural learning of time-dependent nonlinear mappings is presented. It exploits the concept of adjoint operators to enable a fast global computation of the network's response to perturbations in all the systems parameters. The importance of the time boundary conditions of the adjoint functions is discussed. An algorithm is presented in which the adjoint sensitivity equations are solved simultaneously (Le., forward in time) along with the nonlinear dynamics of the neural networks. This methodology makes real-time applications and hardware implementation of temporal learning feasible.