Part of Advances in Neural Information Processing Systems 8 (NIPS 1995)
Brendan J. Frey, Geoffrey E. Hinton, Peter Dayan
The wake-sleep algorithm (Hinton, Dayan, Frey and Neal 1995) is a rel(cid:173) atively efficient method of fitting a multilayer stochastic generative model to high-dimensional data. In addition to the top-down connec(cid:173) tions in the generative model, it makes use of bottom-up connections for approximating the probability distribution over the hidden units given the data, and it trains these bottom-up connections using a simple delta rule. We use a variety of synthetic and real data sets to compare the per(cid:173) formance of the wake-sleep algorithm with Monte Carlo and mean field methods for fitting the same generative model and also compare it with other models that are less powerful but easier to fit.