Kernel Maximum Entropy Data Transformation and an Enhanced Spectral Clustering Algorithm

Part of Advances in Neural Information Processing Systems 19 (NIPS 2006)

Authors

Robert Jenssen, Torbjørn Eltoft, Mark Girolami, Deniz Erdogmus

Abstract

We propose a new kernel-based data transformation technique. It is founded on the principle of maximum entropy (MaxEnt) preservation, hence named kernel MaxEnt. The key measure is Renyi's entropy estimated via Parzen windowing. We show that kernel MaxEnt is based on eigenvectors, and is in that sense similar to kernel PCA, but may produce strikingly different transformed data sets. An enhanced spectral clustering algorithm is proposed, by replacing kernel PCA by kernel MaxEnt as an intermediate step. This has a major impact on performance.