Part of Advances in Neural Information Processing Systems 2 (NIPS 1989)
William Lincoln, Josef Skrzypek
The properties of a cluster of multiple back-propagation (BP) networks are examined and compared to the performance of a single BP net(cid:173) work. The underlying idea is that a synergistic effect within the cluster improves the perfonnance and fault tolerance. Five networks were ini(cid:173) tially trained to perfonn the same input-output mapping. Following training, a cluster was created by computing an average of the outputs generated by the individual networks. The output of the cluster can be used as the desired output during training by feeding it back to the indi(cid:173) vidual networks. In comparison to a single BP network, a cluster of multiple BP's generalization and significant fault tolerance. It appear that cluster advantage follows from simple maxim "you can fool some of the single BP's in a cluster all of the time but you cannot fool all of them all of the time" {Lincoln}