Part of Advances in Neural Information Processing Systems 5 (NIPS 1992)
L. Y. Pratt
Previously, we have introduced the idea of neural network transfer, where learning on a target problem is sped up by using the weights obtained from a network trained for a related source task. Here, we present a new algorithm. called Discriminability-Based Transfer (DBT), which uses an information measure to estimate the utility of hyperplanes defined by source weights in the target network, and rescales transferred weight magnitudes accordingly. Several experiments demonstrate that target networks initialized via DBT learn significantly faster than networks initialized randomly.