Part of Advances in Neural Information Processing Systems 25 (NIPS 2012)
Zhihua Zhang, Bojun Tu
In this paper we study sparsity-inducing nonconvex penalty functions using L´evy processes. We define such a penalty as the Laplace exponent of a subordina- tor. Accordingly, we propose a novel approach for the construction of sparsity- inducing nonconvex penalties. Particularly, we show that the nonconvex logarith- mic (LOG) and exponential (EXP) penalty functions are the Laplace exponents of Gamma and compound Poisson subordinators, respectively. Additionally, we explore the concave conjugate of nonconvex penalties. We find that the LOG and EXP penalties are the concave conjugates of negative Kullback-Leiber (KL) dis- tance functions. Furthermore, the relationship between these two penalties is due to asymmetricity of the KL distance.