Ensemble Learning and Linear Response Theory for ICA

Part of Advances in Neural Information Processing Systems 13 (NIPS 2000)

Bibtex Metadata Paper


Pedro Højen-Sørensen, Ole Winther, Lars Hansen


We propose a general Bayesian framework for performing independent component analysis (leA) which relies on ensemble learning and lin(cid:173) ear response theory known from statistical physics. We apply it to both discrete and continuous sources. For the continuous source the underde(cid:173) termined (overcomplete) case is studied. The naive mean-field approach fails in this case whereas linear response theory-which gives an improved estimate of covariances-is very efficient. The examples given are for sources without temporal correlations. However, this derivation can eas(cid:173) ily be extended to treat temporal correlations. Finally, the framework offers a simple way of generating new leA algorithms without needing to define the prior distribution of the sources explicitly.