Part of Advances in Neural Information Processing Systems 18 (NIPS 2005)
Active learning is the problem in supervised learning to design the loca- tions of training input points so that the generalization error is minimized. Existing active learning methods often assume that the model used for learning is correctly speciﬁed, i.e., the learning target function can be ex- pressed by the model at hand. In many practical situations, however, this assumption may not be fulﬁlled. In this paper, we ﬁrst show that the ex- isting active learning method can be theoretically justiﬁed under slightly weaker condition: the model does not have to be correctly speciﬁed, but slightly misspeciﬁed models are also allowed. However, it turns out that the weakened condition is still restrictive in practice. To cope with this problem, we propose an alternative active learning method which can be theoretically justiﬁed for a wider class of misspeciﬁed models. Thus, the proposed method has a broader range of applications than the exist- ing method. Numerical studies show that the proposed active learning method is robust against the misspeciﬁcation of models and is thus reli- able.
Let us discuss the regression problem of learning a real-valued functionfx deﬁned on Rd from training examplesfxi;yijyi=fxi(cid:15)igi=1; wheref(cid:15)igi=1 are i.i.d. noise with mean zero and unknown variance(cid:27)2 bfx=Xi=1(cid:11)iix; wherefixgi=1 are ﬁxed linearly independent functions and(cid:11)=(cid:11)1;(cid:11)2;:::;(cid:11)> We evaluate the goodness of the learned functionbfx by the expected squared test error are drawn independently from a distribution with densityx, the generalization error is G=E(cid:15)Z(cid:16)bfxfx(cid:17)2xdx;
over test input points and noise (i.e., the generalization error). When the test input points
Introduction and Problem Formulation
are parameters to be learned.
lowing linear regression model for learning.