{"title": "Shaping the State Space Landscape in Recurrent Networks", "book": "Advances in Neural Information Processing Systems", "page_first": 105, "page_last": 112, "abstract": null, "full_text": "Shaping the State Space Landscape in Recurrent \n\nNetworks \n\nPatrice Y. Simard >I< \nComputer Science Dept. \nUniversity of Rochester \nRochester, NY 14627 \n\nJean Pierre Raysz \n\nLIUC \n\nU niversite de Caen \n14032 Caen Cedex \n\nFrance \n\nAbstract \n\nBernard Victorri \nELSAP \nUniversite de Caen \n14032 Caen Cedex \nFrance \n\nFully recurrent (asymmetrical) networks can be thought of as dynamic \nsystems. The dynamics can be shaped to perform content addressable \nmemories, recognize sequences, or generate trajectories. Unfortunately \nseveral problems can arise: First, the convergence in the state space is \nnot guaranteed. Second, the learned fixed points or trajectories are not \nnecessarily stable. Finally, there might exist spurious fixed points and/or \nspurious \"attracting\" trajectories that do not correspond to any patterns. \nIn this paper, we introduce a new energy function that presents solutions \nto all of these problems. We present an efficient gradient descent algorithm \nwhich directly acts on the stability of the fixed points and trajectories and \non the size and shape of the corresponding basin and valley of attraction. \nThe results are illustrated by the simulation of a small content addressable \nmemory. \n\n1 \n\nINTRODUCTION \n\nRecurrent neural networks have the capability of storing information in the state \nof their units. The temporal evolution of these states constitutes the dynamics of \nthe system and depends on the weights and the input of the network. In the case \nof symmetric connections, the dynamics have been shown to be convergent [2] and \nvarious procedures are known for finding the weights to compute different tasks. \nIn unconstrained neural networks however, little is known about how to train the \nweights of the network when the convergence of the dynamics is not guaranteed. \nIn his review paper [1], Hirsh defines the conditions which must be satisfied for \n\n>l