Part of Advances in Neural Information Processing Systems 18 (NIPS 2005)
Ross Lippert, Ryan Rifkin
We consider regularized least-squares (RLS) with a Gaussian kernel. We prove that if we let the Gaussian bandwidth σ → ∞ while letting the regularization parameter λ → 0, the RLS solution tends to a polynomial whose order is controlled by the rielative rates of decay of 1 σ2 and λ: if λ = σ−(2k+1), then, as σ → ∞, the RLS solution tends to the kth order polynomial with minimal empirical error. We illustrate the result with an example.