Part of Advances in Neural Information Processing Systems 10 (NIPS 1997)
Don Hush, Fernando Lozano, Bill Horne
We present a computationally efficient algorithm for function ap(cid:173) proximation with piecewise linear sigmoidal nodes. A one hidden layer network is constructed one node at a time using the method of fitting the residual. The task of fitting individual nodes is accom(cid:173) plished using a new algorithm that searchs for the best fit by solving a sequence of Quadratic Programming problems. This approach of(cid:173) fers significant advantages over derivative-based search algorithms (e.g. backpropagation and its extensions). Unique characteristics of this algorithm include: finite step convergence, a simple stop(cid:173) ping criterion, a deterministic methodology for seeking "good" local minima, good scaling properties and a robust numerical implemen(cid:173) tation.