Online Independent Component Analysis with Local Learning Rate Adaptation

Part of Advances in Neural Information Processing Systems 12 (NIPS 1999)

Bibtex Metadata Paper

Authors

Nicol Schraudolph, Xavier Giannakopoulos

Abstract

Stochastic meta-descent (SMD) is a new technique for online adap(cid:173) tation of local learning rates in arbitrary twice-differentiable sys(cid:173) tems. Like matrix momentum it uses full second-order information while retaining O(n) computational complexity by exploiting the efficient computation of Hessian-vector products. Here we apply SMD to independent component analysis, and employ the result(cid:173) ing algorithm for the blind separation of time-varying mixtures. By matching individual learning rates to the rate of change in each source signal's mixture coefficients, our technique is capable of si(cid:173) multaneously tracking sources that move at very different, a priori unknown speeds.