Part of Advances in Neural Information Processing Systems 5 (NIPS 1992)
Bill Baird, Todd Troyer, Frank Eeckman
We have designed an architecture to span the gap between bio(cid:173) physics and cognitive science to address and explore issues of how a discrete symbol processing system can arise from the continuum, and how complex dynamics like oscillation and synchronization can then be employed in its operation and affect its learning. We show how a discrete-time recurrent "Elman" network architecture can be constructed from recurrently connected oscillatory associative memory modules described by continuous nonlinear ordinary dif(cid:173) ferential equations. The modules can learn connection weights be(cid:173) tween themselves which will cause the system to evolve under a clocked "machine cycle" by a sequence of transitions of attractors within the modules, much as a digital computer evolves by transi(cid:173) tions of its binary flip-flop attractors. The architecture thus em(cid:173) ploys the principle of "computing with attractors" used by macro(cid:173) scopic systems for reliable computation in the presence of noise. We have specifically constructed a system which functions as a finite state automaton that recognizes or generates the infinite set of six symbol strings that are defined by a Reber grammar. It is a symbol processing system, but with analog input and oscillatory subsym(cid:173) bolic representations. The time steps (machine cycles) of the sys(cid:173) tem are implemented by rhythmic variation (clocking) of a bifurca(cid:173) tion parameter. This holds input and "context" modules clamped at their attractors while 'hidden and output modules change state, then clamps hidden and output states while context modules are released to load those states as the new context for the next cycle of input. Superior noise immunity has been demonstrated for systems with dynamic attractors over systems with static attractors, and synchronization ("binding") between coupled oscillatory attractors in different modules has been shown to be important for effecting reliable transitions.