Part of Advances in Neural Information Processing Systems 1 (NIPS 1988)
Patrice Simard, Mary Ottaway, Dana Ballard
This paper provides a systematic analysis of the recurrent backpropaga(cid:173) tion (RBP) algorithm, introducing a number of new results. The main limitation of the RBP algorithm is that it assumes the convergence of the network to a stable fixed point in order to backpropagate the error signals. We show by experiment and eigenvalue analysis that this condi(cid:173) tion can be violated and that chaotic behavior can be avoided. Next we examine the advantages of RBP over the standard backpropagation al(cid:173) gorithm. RBP is shown to build stable fixed points corresponding to the input patterns. This makes it an appropriate tool for content address(cid:173) able memories, one-to-many function learning, and inverse problems.