Part of Advances in Neural Information Processing Systems 12 (NIPS 1999)
Gal Chechik, Isaac Meilijson, Eytan Ruppin
This paper revisits the classical neuroscience paradigm of Hebbian learning. We find that a necessary requirement for effective as(cid:173) sociative memory learning is that the efficacies of the incoming synapses should be uncorrelated. This requirement is difficult to achieve in a robust manner by Hebbian synaptic learning, since it depends on network level information. Effective learning can yet be obtained by a neuronal process that maintains a zero sum of the in(cid:173) coming synaptic efficacies. This normalization drastically improves the memory capacity of associative networks, from an essentially bounded capacity to one that linearly scales with the network's size. It also enables the effective storage of patterns with heterogeneous coding levels in a single network. Such neuronal normalization can be successfully carried out by activity-dependent homeostasis of the neuron's synaptic efficacies, which was recently observed in cortical tissue. Thus, our findings strongly suggest that effective associa(cid:173) tive learning with Hebbian synapses alone is biologically implausi(cid:173) ble and that Hebbian synapses must be continuously remodeled by neuronally-driven regulatory processes in the brain.