Function Approximation with the Sweeping Hinge Algorithm

Part of Advances in Neural Information Processing Systems 10 (NIPS 1997)

Bibtex Metadata Paper

Authors

Don Hush, Fernando Lozano, Bill Horne

Abstract

We present a computationally efficient algorithm for function ap(cid:173) proximation with piecewise linear sigmoidal nodes. A one hidden layer network is constructed one node at a time using the method of fitting the residual. The task of fitting individual nodes is accom(cid:173) plished using a new algorithm that searchs for the best fit by solving a sequence of Quadratic Programming problems. This approach of(cid:173) fers significant advantages over derivative-based search algorithms (e.g. backpropagation and its extensions). Unique characteristics of this algorithm include: finite step convergence, a simple stop(cid:173) ping criterion, a deterministic methodology for seeking "good" local minima, good scaling properties and a robust numerical implemen(cid:173) tation.