Part of Advances in Neural Information Processing Systems 8 (NIPS 1995)
Mark Craven, Jude Shavlik
A significant limitation of neural networks is that the represen(cid:173) tations they learn are usually incomprehensible to humans. We present a novel algorithm , TREPAN, for extracting comprehensible, symbolic representations from trained neural networks. Our algo(cid:173) rithm uses queries to induce a decision tree that approximates the concept represented by a given network. Our experiments demon(cid:173) strate that TREPAN is able to produce decision trees that maintain a high level of fidelity to their respective networks while being com(cid:173) prehensible and accurate. Unlike previous work in this area, our algorithm is general in its applicability and scales well to large net(cid:173) works and problems with high-dimensional input spaces.