Part of Advances in Neural Information Processing Systems 11 (NIPS 1998)
Manfred Opper, Ole Winther
We discuss the application of TAP mean field methods known from the Statistical Mechanics of disordered systems to Bayesian classifi(cid:173) cation models with Gaussian processes. In contrast to previous ap(cid:173) proaches, no knowledge about the distribution of inputs is needed. Simulation results for the Sonar data set are given.
1 Modeling with Gaussian Processes
Bayesian models which are based on Gaussian prior distributions on function spaces are promising non-parametric statistical tools. They have been recently introduced into the Neural Computation community (Neal 1996, Williams & Rasmussen 1996, Mackay 1997). To give their basic definition, we assume that the likelihood of the output or target variable T for a given input s E RN can be written in the form p(Tlh(s)) where h : RN --+ R is a priori assumed to be a Gaussian random field. If we assume fields with zero prior mean, the statistics of h is entirely defined by the second order correlations C(s, S') == E[h(s)h(S')], where E denotes expectations
310
M Opper and 0. Winther
with respect to the prior. Interesting examples are