Part of Neural Information Processing Systems 0 (NIPS 1987)
James Keeler
The information capacity of Kanerva's Sparse, Distributed Memory (SDM) and Hopfield-type neural networks is investigated. Under the approximations used here, it is shown that the to(cid:173) tal information stored in these systems is proportional to the number connections in the net(cid:173) work. The proportionality constant is the same for the SDM and HopJreld-type models in(cid:173) dependent of the particular model, or the order of the model. The approximations are checked numerically. This same analysis can be used to show that the SDM can store se(cid:173) quences of spatiotemporal patterns, and the addition of time-delayed connections allows the retrieval of context dependent temporal patterns. A minor modification of the SDM can be used to store correlated patterns.