NeurIPS 2020

HAWQ-V2: Hessian Aware trace-Weighted Quantization of Neural Networks

Meta Review

This paper on the surface makes a small change of an existing approach to base neural network weight quantization on the Hessian trace instead of max eigenvalue. This is motivated by the observation that the Hessian should be positive definite if the network is optimized to a local optimum. This change, although superficially minor compared with using the largest eigenvalue (as proposed in [7]) leads to the ability to apply Hutchinson's algorithm to compute an efficient sample based approximation to the trace, which leads to large computational speedups and modest performance improvements. The reviewers were unanimous that the paper is at least marginally above the acceptance threshold.