Part of Advances in Neural Information Processing Systems 8 (NIPS 1995)
René Gourley
Harmony networks have been proposed as a means by which con(cid:173) nectionist models can perform symbolic computation. Indeed, pro(cid:173) ponents claim that a harmony network can be built that constructs parse trees for strings in a context free language. This paper shows that harmony networks do not work in the following sense: they construct many outputs that are not valid parse trees.
In order to show that the notion of systematicity is compatible with connectionism, Paul Smolensky, Geraldine Legendre and Yoshiro Miyata (Smolensky, Legendre, and Miyata 1992; Smolen sky 1993; Smolen sky, Legendre, and Miyata 1994) pro(cid:173) posed a mechanism, "Harmony Theory," by which connectionist models purportedly perform structure sensitive operations without implementing classical algorithms. Harmony theory describes a "harmony network" which, in the course of reaching a stable equilibrium, apparently computes parse trees that are valid according to the rules of a particular context-free grammar.
Harmony networks consist of four major components which will be explained in detail in Section 1. The four components are,
Tensor Representation: A means to interpret the activation vector of a connec(cid:173)
tionist system as a parse tree for a string in a context-free language.
Harmony: A function that maps all possible parse trees to the non-positive inte(cid:173)
gers so that a parse tree is valid if and only if its harmony is zero.
Energy: A function that maps the set of activation vectors to the real numbers
and which is minimized by certain connectionist networks!.
Recursive Construction: A system for determining the weight matrix of a con(cid:173)
nectionist network so that if its activation vector is interpreted as a parse
1 Smolensky, Legendre and Miyata use the term "harmony" to refer to both energy and harmony. To distinguish between them, we will use the term that is often used to describe the Lyapunov function of dynamic systems, "energy" (see for example Golden 1986).