Max Welling, Richard Zemel, Geoffrey E. Hinton
Boosting algorithms and successful applications thereof abound for clas- siﬁcation and regression learning problems, but not for unsupervised learning. We propose a sequential approach to adding features to a ran- dom ﬁeld model by training them to improve classiﬁcation performance between the data and an equal-sized sample of “negative examples” gen- erated from the model’s current estimate of the data density. Training in each boosting round proceeds in three stages: ﬁrst we sample negative examples from the model’s current Boltzmann distribution. Next, a fea- ture is trained to improve classiﬁcation performance between data and negative examples. Finally, a coefﬁcient is learned which determines the importance of this feature relative to ones already in the pool. Negative examples only need to be generated once to learn each new feature. The validity of the approach is demonstrated on binary digits and continuous synthetic data.