Part of Advances in Neural Information Processing Systems 12 (NIPS 1999)
Christophe Andrieu, João de Freitas, Arnaud Doucet
In this paper, we propose a full Bayesian model for neural networks. This model treats the model dimension (number of neurons), model parameters, regularisation parameters and noise parameters as ran(cid:173) dom variables that need to be estimated. We then propose a re(cid:173) versible jump Markov chain Monte Carlo (MCMC) method to per(cid:173) form the necessary computations. We find that the results are not only better than the previously reported ones, but also appear to be robust with respect to the prior specification. Moreover, we present a geometric convergence theorem for the algorithm.