Part of Advances in Neural Information Processing Systems 1 (NIPS 1988)
Raoul Tawel
An improved learning paradigm that offers a significant reduction in com(cid:173) putation time during the supervised learning phase is described. It is based on extending the role that the neuron plays in artificial neural systems. Prior work has regarded the neuron as a strictly passive, non-linear processing element, and the synapse on the other hand as the primary source of information processing and knowledge retention. In this work, the role of the neuron is extended insofar as allow(cid:173) ing its parameters to adaptively participate in the learning phase. The temperature of the sigmoid function is an example of such a parameter. During learning, both the synaptic interconnection weights w[j and the neuronal temperatures Tr are opti(cid:173) mized so as to capture the knowledge contained within the training set. The method allows each neuron to possess and update its own characteristic local temperature. This algorithm has been applied to logic type of problems such as the XOR or parity problem, resulting in a significant decrease in the required number of training cycles.