Part of Advances in Neural Information Processing Systems 4 (NIPS 1991)
Jenq-Neng Huang, Hang Li, Martin Maechler, R. Martin, Jim Schimert
Two projection based feedforward network learning methods for model(cid:173) free regression problems are studied and compared in this paper: one is the popular back-propagation learning (BPL); the other is the projection pursuit learning (PPL). Unlike the totally parametric BPL method, the PPL non-parametrically estimates unknown nonlinear functions sequen(cid:173) tially (neuron-by-neuron and layer-by-Iayer) at each iteration while jointly estimating the interconnection weights. In terms of learning efficiency, both methods have comparable training speed when based on a Gauss(cid:173) Newton optimization algorithm while the PPL is more parsimonious. In terms of learning robustness toward noise outliers, the BPL is more sensi(cid:173) tive to the outliers.