Part of Advances in Neural Information Processing Systems 8 (NIPS 1995)
Christopher Williams, Carl Rasmussen
The Bayesian analysis of neural networks is difficult because a sim(cid:173) ple prior over weights implies a complex prior distribution over functions . In this paper we investigate the use of Gaussian process priors over functions, which permit the predictive Bayesian anal(cid:173) ysis for fixed values of hyperparameters to be carried out exactly using matrix operations. Two methods, using optimization and av(cid:173) eraging (via Hybrid Monte Carlo) over hyperparameters have been tested on a number of challenging problems and have produced excellent results.