Asymptotic Universality for Learning Curves of Support Vector Machines

Part of Advances in Neural Information Processing Systems 14 (NIPS 2001)

Authors

Manfred Opper, Robert Urbanczik

Abstract

Using methods of Statistical Physics, we investigate the rOle of model complexity in learning with support vector machines (SVMs). We show the advantages of using SVMs with kernels of infinite complexity on noisy target rules, which, in contrast to common theoretical beliefs, are found to achieve optimal general(cid:173) ization error although the training error does not converge to the generalization error. Moreover, we find a universal asymptotics of the learning curves which only depend on the target rule but not on the SVM kernel.