Part of Advances in Neural Information Processing Systems 15 (NIPS 2002)
Max Welling, Simon Osindero, Geoffrey E. Hinton
We propose a model for natural images in which the probability of an im- age is proportional to the product of the probabilities of some filter out- puts. We encourage the system to find sparse features by using a Student- t distribution to model each filter output. If the t-distribution is used to model the combined outputs of sets of neurally adjacent filters, the sys- tem learns a topographic map in which the orientation, spatial frequency and location of the filters change smoothly across the map. Even though maximum likelihood learning is intractable in our model, the product form allows a relatively efficient learning procedure that works well even for highly overcomplete sets of filters. Once the model has been learned it can be used as a prior to derive the “iterated Wiener filter” for the pur- pose of denoising images.