Part of Advances in Neural Information Processing Systems 15 (NIPS 2002)
Herbert Jaeger
Echo state networks (ESN) are a novel approach to recurrent neu(cid:173) ral network training. An ESN consists of a large, fixed, recurrent "reservoir" network, from which the desired output is obtained by training suitable output connection weights. Determination of op(cid:173) timal output weights becomes a linear, uniquely solvable task of MSE minimization. This article reviews the basic ideas and de(cid:173) scribes an online adaptation scheme based on the RLS algorithm known from adaptive linear systems. As an example, a 10-th or(cid:173) der NARMA system is adaptively identified. The known benefits of the RLS algorithms carryover from linear systems to nonlinear ones; specifically, the convergence rate and misadjustment can be determined at design time.