Part of Advances in Neural Information Processing Systems 30 (NIPS 2017)
Jalil Kazemitabar, Arash Amini, Adam Bloniarz, Ameet S. Talwalkar
Decision trees and random forests are well established models that not only offer good predictive performance, but also provide rich feature importance information. While practitioners often employ variable importance methods that rely on this impurity-based information, these methods remain poorly characterized from a theoretical perspective. We provide novel insights into the performance of these methods by deriving finite sample performance guarantees in a high-dimensional setting under various modeling assumptions. We further demonstrate the effectiveness of these impurity-based methods via an extensive set of simulations.