Part of Advances in Neural Information Processing Systems 15 (NIPS 2002)
Max Welling, Simon Osindero, Geoffrey E. Hinton
We propose a model for natural images in which the probability of an im- age is proportional to the product of the probabilities of some ﬁlter out- puts. We encourage the system to ﬁnd sparse features by using a Student- t distribution to model each ﬁlter output. If the t-distribution is used to model the combined outputs of sets of neurally adjacent ﬁlters, the sys- tem learns a topographic map in which the orientation, spatial frequency and location of the ﬁlters change smoothly across the map. Even though maximum likelihood learning is intractable in our model, the product form allows a relatively efﬁcient learning procedure that works well even for highly overcomplete sets of ﬁlters. Once the model has been learned it can be used as a prior to derive the “iterated Wiener ﬁlter” for the pur- pose of denoising images.