Part of Advances in Neural Information Processing Systems 10 (NIPS 1997)
Yuansheng Xiong, Chulan Kwon, Jong-Hoon Oh
We study the storage capacity of a fully-connected committee ma(cid:173) chine with a large number K of hidden nodes. The storage capac(cid:173) ity is obtained by analyzing the geometrical structure of the weight space related to the internal representation . By examining the as(cid:173) ymptotic behavior of order parameters in the limit of large K, the storage capacity Q c is found to be proportional to ]{ Jln ]{ up to the leading order. This result satisfies the mathematical bound given by Mitchison and Durbin , whereas the replica-symmetric solution in a conventional Gardner's approach violates this bound.