Part of Advances in Neural Information Processing Systems 5 (NIPS 1992)
Barry Flower, Marwan Jabri
The algorithm presented performs gradient descent on the weight space of an Artificial Neural Network (ANN), using a finite difference to approximate the gradient The method is novel in that it achieves a com(cid:173) putational complexity similar to that of Node Perturbation, O(N3), but does not require access to the activity of hidden or internal neurons. This is possible due to a stochastic relation between perturbations at the weights and the neurons of an ANN. The algorithm is also similar to Weight Perturbation in that it is optimal in terms of hardware require(cid:173) ments when used for the training ofVLSI implementations of ANN's.