Part of Advances in Neural Information Processing Systems 6 (NIPS 1993)
I. M. Elfadel, J. L. Wyatt, Jr.
We use mean-field theory methods from Statistical Mechanics to derive the "softmax" nonlinearity from the discontinuous winner(cid:173) take-all (WTA) mapping. We give two simple ways of implementing "soft max" as a multiterminal network element. One of these has a number of important network-theoretic properties. It is a recipro(cid:173) cal, passive, incrementally passive, nonlinear, resistive multitermi(cid:173) nal element with a content function having the form of information(cid:173) theoretic entropy. These properties should enable one to use this element in nonlinear RC networks with such other reciprocal el(cid:173) ements as resistive fuses and constraint boxes to implement very high speed analog optimization algorithms using a minimum of hardware.