Part of Advances in Neural Information Processing Systems 7 (NIPS 1994)
Justin Boyan, Andrew Moore
A straightforward approach to the curse of dimensionality in re(cid:173) inforcement learning and dynamic programming is to replace the lookup table with a generalizing function approximator such as a neu(cid:173) ral net. Although this has been successful in the domain of backgam(cid:173) mon, there is no guarantee of convergence. In this paper, we show that the combination of dynamic programming and function approx(cid:173) imation is not robust, and in even very benign cases, may produce an entirely wrong policy. We then introduce Grow-Support, a new algorithm which is safe from divergence yet can still reap the benefits of successful generalization .