{"title": "An Efficient Clustering Algorithm Using Stochastic Association Model and Its Implementation Using Nanostructures", "book": "Advances in Neural Information Processing Systems", "page_first": 1115, "page_last": 1122, "abstract": null, "full_text": "An Ef\ufb01cient Clustering Algorithm Using\n\nStochastic Association Model and Its\nImplementation Using Nanostructures\n\nTakashi Morie, Tomohiro Matsuura, Makoto Nagata, and Atsushi Iwata\n\nGraduate School of Advanced Sciences of Matter, Hiroshima University\n\nHigashi-hiroshima, 739-8526 Japan.\n\nhttp://www.dsl.hiroshima-u.ac.jp\n\nmorie@dsl.hiroshima-u.ac.jp\n\nAbstract\n\nThis paper describes a clustering algorithm for vector quantizers using a\n\u201cstochastic association model\u201d. It offers a new simple and powerful soft-\nmax adaptation rule. The adaptation process is the same as the on-line\nK-means clustering method except for adding random \ufb02uctuation in the\ndistortion error evaluation process. Simulation results demonstrate that\nthe new algorithm can achieve ef\ufb01cient adaptation as high as the \u201cneural\ngas\u201d algorithm, which is reported as one of the most ef\ufb01cient clustering\nmethods. It is a key to add uncorrelated random \ufb02uctuation in the simi-\nlarity evaluation process for each reference vector. For hardware imple-\nmentation of this process, we propose a nanostructure, whose operation\nis described by a single-electron circuit. It positively uses \ufb02uctuation in\nquantum mechanical tunneling processes.\n\n1 Introduction\n\nthe average distortion error:\n\nVector quantization (VQ) techniques are used in a wide range of applications, including\nspeech and image processing, data compression. VQ techniques encode a data manifold\n\n\u0002\u0001\u0004\u0003\u0006\u0005 using only a \ufb01nite set of reference vectors\u0007\t\b\u000b\n\f\u0007\u000e\r\u0010\u000f\u0012\u0011\u0013\u0011\u0012\u0011\u0012\u000f\u0014\u0007\u0016\u0015\u0018\u0017 . A data vector\u0019\u001b\u001a\nis represented by the best-matching or \u201cwinning\u201d reference vector\u0007\u001d\u001c , which minimizes\nwhere'\n\nis the probability distribution of data vectors over manifold\n\nVarious clustering algorithms to obtain the best reference vectors have been reported. Here,\nwe treat on-line training, in which the data point distribution is not given a priori, but instead\na stochastic sequence of incoming sample data points drives the adaptation procedure.\n\n &%('\n\n\u0005-,\n\n\f\u0019)\u0017+*\n\n\u001f! \n\n\u0019#\"$\u0007\n\n.\u0019)\u0017\n\n(1)\n\n.\n\nThe straightforward approach is the well-known on-line K-means clustering algorithm, in\nwhich only the nearest reference vector to the sample vector is adjusted;\n\n\u0007\u001d01\b324\u0011\u00125\u00120\n\n\u00116\n\f\u00197\n.8+\u00179\"$\u0007\u00160:\u0017&\u000f\n\n(2)\n\n\n\u001e\n\b\n\u001c\n\u000f\n/\n\u001c\n\fwhere parameter\n\nde\ufb01nes the proximity.\n\n\b324\u0011\n\n\u0007\n\t\f\u000b\u000e\r\n\u0007\u0014\u0013\u0015\u000b\n\n\u0002\u0004\u0003\u0006\u0005\b\u0007\n\u0011\u00101\n\n\u0003\u0012\u0005\u0012\u0007\n\n\f\u0019\n\n.8+\u00179\"$\u0007\n\n\u0017&\u000f\n\n(3)\n\n(5)\n\nis the step size and 5\u00120\u0001\n\nis the Kronecker delta. However, this simple clustering\nalgorithm is often stuck in a local minimum. To avoid this dif\ufb01culty, a common approach\nis to introduce a \u201csoft-max\u201d adaptation rule that not only adjusts the \u201cwinning\u201d reference\n\nwhere, 2\nvector but affects other reference vectors depending on their proximity to\u0019\nThe maximum-entropy (ME) algorithm [1] adjusts all reference vectors \u0007\u00160 depending on\nthe Euclidean distance to\u0019\n\n;\n\n.\n\nThe Kohonen\u2019s self-organization map (SOM) algorithm [2] is another well-known model;\n(4)\nIn this model, every reference vector is assigned to a site of a lattice. Each time a sample\nvector is presented, not only the \u201cwinning\u201d reference vector is adjusted but also the refer-\nence vectors assigned to the lattice sites adjacent to the winner are updated according to\nfunction\n\n\u0007\u001601\b324\u0011\u0018\u0017\u0012\u0019\n\n\f8+\u00179\"$\u0007\u00160\n\n\u000f\u001c\u001b\u0012\u0017\n\n\u0017\u001e\u001d\n\n\f\u001a\n\n.\u0019\n\n\u0017\u0006\u0019\n\n\u000e\u001a\n\n\u000f\u001c\u001b\u0012\u0017 , which is typically chosen to be a Gaussian:\n\r\u001c\u000b\n\n\u0003 \u001f\n\n\u001c\"\n\n\u0007\u0014\t\n\n\u0007\n!\n\n\u0017\u0012\u0019\n\n\f\u001a\n\n\u000f\u0011\u001b\u0012\u0017\n\nwhere\n\nis a parameter that de\ufb01nes the proximity.\n\nThe neural-gas (NG) clustering algorithm [3] is a powerful soft-max adaptation rule, in\nwhich all reference vectors are adjusted depending on the \u201cneighborhood ranking\u201d;\n\nis typically as follows:\n\nwhere\n\n(&\n\n0\u0014\n.\u0019\n\n\u000f+\u0007#\u0017\n\n\u0011$\u0017\u0006%\n\n\u0007\u001d0)\b\u00042\n\n\u000f+\u0007\u000e\u0017+\u00179\u00116\n.\u00197\n\f8+\u0017\nis the ranking, which depends on \u0019\n\u0003+*\n\n'&\n\n(&\n\n0\u0014\n.\u0019\n\u00177\b\n\n\u0007\u00160\n\nand the whole set \u0007\n\n(7)\nwhere parameter\nde\ufb01nes the proximity. This algorithm exhibits faster convergence to\nsmaller distortion errors, however consumes higher computational power especially for\nsorting. An ef\ufb01cient version of the NG clustering that adjusts only several reference vectors\nhaving upper ranking was also proposed [4].\n\n\u0017)%\n\n(6)\n. The function\n\nIn the next section, we propose a new ef\ufb01cient soft-max adaptation algorithm.\nIt em-\nploys the stochastic association model that we have proposed related to single-electron\ncircuits [5], [6]. In Sec. 3, it is demonstrated from simulation results that this new cluster-\ning algorithm is as powerful as the other algorithms. In Sec. 4, we propose a nanostructure\nbased on a single-electron circuit for implementing the stochastic association model.\n\n2 Stochastic association algorithm\n\nA usual associative memory is de\ufb01ned as a system that deterministically extracts the vector\nmost similar to the input vector from the stored reference vectors. This just corresponds to\nthe process choosing the winning reference vector for a certain data vector in all conven-\ntional clustering algorithms.\n\nIn our stochastic association (SA) model, the association probability depends on the simi-\nlarity between the input and the reference vectors. The SA algorithm extracts not only the\nreference vector most similar to the input but also other similar reference vectors with the\nprobability depending on the similarity.\n\nIn the SA algorithm, stochastic \ufb02uctuation is added in the evaluation process of distortion\nerror\n\n0 between data vector\u0019 and reference vector\u0007\u00160 . We propose this algorithm inspired\n\n/\n\u0007\n0\n\u0019\n\u0003\n\u000f\n\u0015\n\u0002\n\u0019\n\u0003\n\n\u0011\n0\n\u0016\n/\n\u0011\n\b\n\u0002\n\u0003\n\u001f\n\u0007\n%\n\u0019\n\u000f\n#\n/\n\"\n\u0017\n\u000f\n&\n\u0017\n%\n0\n\u0017\n\u0002\n\"\n%\n\u000f\n,\n-\n\fPr\n\n(r -D )n\n\nn\n\nRi\n\nRn\n\nA (r\n\n)n\n \n\ni\n\n\u0001\u0001\n\u0001\u0001\n\u0001\u0001\n\u0001\u0001\n\nrn\n\nDn\nwn\n\nDi\nwi\n\nDistance\n\nFigure 1: Probability distribution in evaluation of the distortion error between the data\nvector and each reference vector.\n\n.\n\n(9)\n\n(10)\n\nfor a\n\n0:\u0017\n\nThe distortion error\n\nby the quantum mechanical property of single-electron circuits as described in Sec. 4, and\n\nwe expect that such \ufb02uctuation helps to avoid getting stuck in local minima of\u001e\ndistance \n\n0 can be the squared Euclidean distance \n . The evaluation result is represented by\nis also considered as a random variable. The probability that \u0002\n\n\u0019#\"\nis a random variable with probability distribution function \u0007\u0006\n\b\u00056\u0017 . Therefore, the\n0 has\n\n% or the Manhattan\n\n0\u0004\u0003\u0006\u0005\n\n\"#\u0007\n\n(8)\n\nis determined by\n\n0)\b\u000f\t\u00130\u0011\u00104\b\u0012\u0007\u0006\n\b\t\u00130\n\u000b\u0016\u0015\u0018\u0017\u001a\u0019\u001c\u001b\n\u001b-\b\u0014\u0013\n\n\u0007\u00160\nwhere \u0005\nevaluation result \u0002\nvalue\t\u00130 is represented by\n\n\f\u000b\u000e\r\nThe winning reference vector\u0007#\u001c\nThe probability that reference vector\u0007\u001e\u001d becomes the winner when\u0002\n\n\b\t\u001f\u001d\u001d\"\ncertain data vector is given by the product of \u0007\n\u000f#\")\u001a%$\ngiven by integrating it with\t\u001f\u001d ;\n*+\t,\u001d-\u0007\u0006\n.\t,\u001d\n\u001b\u0006\b\u000f&(\u0010\n\n.\t,\u001d\n\u0007\u0006\n.\t\u0018\"\n\n\u001d has value\t\u001f\u001d\n\u0017 and the probability that \u0002\n\b'& as shown in Fig. 1. Therefore, the probability that \u0007\n\u00170/\n0#1\n\u00100\u001d\n\n0! \n\u001d becomes the winner is\n\n.\t,\u001d\n\nIf the winning reference vector is updated as expressed by eq. (2), the SA model can provide\na new soft-max adaptation rule. Figure 2 shows an architecture for clustering processing\nusing the SA model. The distortion error between the input vector and each stored refer-\nence vector is evaluated in parallel with stochastic \ufb02uctuation. The winner-take-all circuit\ndeterministically extracts the winner, and the winning reference vector is only updated with\na constant value. As in the K-means algorithm, only one reference vector is adjusted for\neach adaptation step and the update value for the selected reference vector is independent of\nsimilarity or proximity. However, unlike the K-means algorithm, the adjusted vector is not\nalways the most similar reference vector, and sometimes other similar vectors are adjusted.\nThe total adjusting tendency in the SA algorithm seems similar to the NG or ME algorithm\nbecause the probability of reference vector selection is determined by the neighborhood\nranking and the distances between each reference vector and a given data vector.\n\n\u001743\n\n0:\u0017(*8\t\n\n\f\u000b\u000e\n\n\u001f*)\n\n-\u001a\u001d\n\n-\u001a\u001d\n\n(11)\n\n(12)\n\n576\n\nj\n-\n\u0019\n0\n \n\u0002\n0\n\b\n-\n\u000f\n0\n\u0002\n\"\n-\n\u001d\n0\n\n\u0002\n0\n\u0010\n\u001d\n\t\n\u001d\n\b\n\u0003\n)\n\"\n2\n0\n\u0017\n2\n0\n\u001f\n)\n-\n\u001d\n\f\u0005\u0001\u0005\n\u0005\u0001\u0005\n\u0005\u0001\u0005\n\u0005\u0001\u0005\n\u0005\u0001\u0005\n\u0005\u0001\u0005\n\u0005\u0001\u0005\n\u0005\u0001\u0005\n\nv\n \nr\no\nt\nc\ne\nv\n \nt\nu\np\nn\nI\n\nupdate only one vector\n\nReference vectors \n\ndistortion error \nevaluation with\n\n\u0002\u0001\u0002\u0001\u0002\u0001\u0002\u0001\u0002\u0001\u0002\u0001\u0002\u0001\u0002\u0001\u0002\u0001\u0002\u0001\u0002\n\u0003\u0001\u0003\u0001\u0003\u0001\u0003\u0001\u0003\u0001\u0003\u0001\u0003\u0001\u0003\u0001\u0003\u0001\u0003\u0001\u0003\u0001\u0003\n\u0004\u0001\u0004\u0001\u0004\u0001\u0004\u0001\u0004\u0001\u0004\u0001\u0004\u0001\u0004\u0001\u0004\u0001\u0004\u0001\u0004\nwi\n\u0002\u0001\u0002\u0001\u0002\u0001\u0002\u0001\u0002\u0001\u0002\u0001\u0002\u0001\u0002\u0001\u0002\u0001\u0002\u0001\u0002\n\u0003\u0001\u0003\u0001\u0003\u0001\u0003\u0001\u0003\u0001\u0003\u0001\u0003\u0001\u0003\u0001\u0003\u0001\u0003\u0001\u0003\u0001\u0003\n\u0004\u0001\u0004\u0001\u0004\u0001\u0004\u0001\u0004\u0001\u0004\u0001\u0004\u0001\u0004\u0001\u0004\u0001\u0004\u0001\u0004\n\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\n\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\n\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\n\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\n\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\n\u0006\u0001\u0006\u0001\u0006\u0001\u0006\u0001\u0006\u0001\u0006\u0001\u0006\u0001\u0006\u0001\u0006\u0001\u0006\u0001\u0006\u0001\u0006\n\u0006\u0001\u0006\u0001\u0006\u0001\u0006\u0001\u0006\u0001\u0006\u0001\u0006\u0001\u0006\u0001\u0006\u0001\u0006\u0001\u0006\u0001\u0006\n\nstochastic fluctuation\n\nWinner-Take-All\n\nwc\n\nFigure 2: Architecture for clustering processing using the SA model.\n\n(a) SA\n\n(b) ME\n\nt=0\n\ntmax = 5000\n\ntmax = 50000\n\nFigure 3: Test problem and clustering results by SA and ME algorithms. Data samples uni-\nformly distribute in square regions, and points represent reference vectors. Both algorithms\nuse the same initial state.\n\n3 Simulation results\n\nIn order to test the performance of the SA algorithm in minimizing the distortion error and\nto compare it with the other soft-max approaches, we performed the same simulation of\nmodel clustering described by Ref. [3]. The data clusters are of square shape within a two-\ndimensional input space as shown in Fig. 3. In the simulation, the number of clusters was\n15, and that of reference vectors was 60. We averaged the results of 50 simulation runs for\neach of which not only the initialization of the reference vectors were chosen randomly but\nalso the 15 clusters were placed randomly.\n\nThe SA algorithm in this simulation used the squared Euclidean distance as a distortion\nerror\nation;\n\n0 and the normal distribution as the probability distribution of the stochastic \ufb02uctu-\n\n(13)\n\n\u0007\u0006\n.\u00056\u00177\b\b\u0007\n\n\t\n\n\u000f\f\u000b\n\n\u000f\u0011\u0010\n\n\u000b\u0013\u0012\u0015\u0014\u0017\u0016\n\n+\"\n\n\u0017\u001e\u001d\n\nx\n-\n%\n\u0017\n3\n\n\u000e\n\u0005\n%\n\u000f\n\u000b\n%\n\f\u0005\u0001\u0005\u0001\u0005\u0001\u0005\u0001\u0005\u0001\u0005\u0001\u0005\u0001\u0005\u0001\u0005\u0001\u0005\u0001\u0005\u0001\u0005\u0001\u0005\u0001\u0005\u0001\u0005\u0001\u0005\u0001\u0005\n\n2\n\n\u0004\u0001\u0004\u0001\u0004\u0001\u0004\u0001\u0004\u0001\u0004\u0001\u0004\u0001\u0004\u0001\u0004\u0001\u0004\u0001\u0004\u0001\u0004\u0001\u0004\u0001\u0004\u0001\u0004\u0001\u0004\u0001\u0004\n\nalgorithm parameter\n\na\n\ne\nc\nn\na\nm\nr\no\nf\nr\ne\nP\n\n\u0003\u0001\u0003\u0001\u0003\u0001\u0003\u0001\u0003\u0001\u0003\u0001\u0003\u0001\u0003\u0001\u0003\u0001\u0003\u0001\u0003\u0001\u0003\u0001\u0003\u0001\u0003\u0001\u0003\u0001\u0003\u0001\u0003\n\n1\n\nSOM\n\nMaximum-entropy(ME)\n\n\u0002\u0001\u0002\u0001\u0002\u0001\u0002\u0001\u0002\u0001\u0002\u0001\u0002\u0001\u0002\u0001\u0002\u0001\u0002\u0001\u0002\u0001\u0002\u0001\u0002\u0001\u0002\u0001\u0002\u0001\u0002\u0001\u0002\n\nStochastic association (SA)\n\nME\nSOM\nNG\nSA\nAll\n\ninitial\n\n\f\u000e\n\n1\n2\n10\n0.2\n0.5\n\n\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\n\nNeural-gas (NG)\n\n0\n\n50000\n\nTotal number of adaptation steps tmax\n\n\ufb01nal\n\n\f\u0010\u000f\n\n10000\n0.01\n0.01\n0.0001\n0.005\n\nFigure 4: Clustering performance of SA algorithm comparing with other clustering meth-\nods. The optimized parameters used in the simulation are also shown.\n\nFigure 3 shows an example of clustering by the SA algorithm compared with that by the\nME algorithm. The result of the SA algorithm demonstrates nearly perfect clustering for\nIn contrast, the clustering result by the ME algorithm is not so good\n\n8\u0017\u0016\u0019\u0018\u001b\u001a\n\n\b\u001d\u001c\u0011\t\n\n\t .\n\nalthough the parameters used were optimized.\n\nHere, all the clustering algorithms including the SA algorithm use an annealing procedure\nto escape local minima. The parameters were gradually reduced during adaptation:\n\n.8+\u00177\b\n\n\u001e \u001f\"!\u000e\u001e\n\n\u0017\u0017#\n\n#%$'&)(\n\n*%+\n\n\u000f\u001c#\n\n\u000f\u001c,)\u000f\f\u000b9\u000f(2\n\n(14)\n\nwhere 8\u0017\u0016\u0019\u0018\u001b\u001a\n\nis the total number of adaptation steps. The values optimized by numerous\npreliminary simulations are shown in Fig. 4, which were used in the simulation described\nhere.\n\n\u00177\"\n\n\f8\u0017\u0016\u0019\u0018\u001b\u001a\n\n\u001e\u0019/10\n\b.-\nrelationships between8\n\nIn order to compare the performance of the algorithms, we used a performance measure\nis the minimal distortion error in this problem. The\nfor the four algorithms are shown in Fig. 4. The cluster-\ning performance of the SA algorithm is nearly equal to that of the NG algorithm, which is\nthe most ef\ufb01cient clustering method in this test problem. The number of adaptation steps\nto reach the steady state and the distortion error at the steady state in the SA algorithm are\nnearly the same as those in the NG algorithm.\n\n\u001e\u0019/ , where \u001e2/\n\u0016\u0019\u0018\u001b\u001a and ,\n\nWe also performed other simulations, one of which was vector quantization of a real image\n\n(\u2018Lena\u2019, 256 3 256 pixels, 8-bit grayscale). In this case, the SOM demonstrated the best\n\nperformance, and the SA algorithm also had the nearly equal performance.\n\nConsequently, comparing with the other soft-max algorithms, the SA algorithm has nearly\nthe best clustering performance. Moreover, it does not require a sorting process unlike the\nNG algorithm nor a searching process of adjacent lattice sites unlike the SOM; only one\nreference vector is adjusted per adaptation step. Thus, the computational power required\nby the SA algorithm is much less than that required by the other soft-max algorithm. If the\n, the total updating steps of reference vectors in the SA\ntimes as many as those in the other algorithms. Thus, the SA algorithm\n\nnumber of reference vectors is \u0007\nalgorithm are \n\nis the most ef\ufb01cient clustering method.\n\n\u0006\n\u0006\n\u0006\n\u0006\n\u0006\n\u0006\n\u0006\n\u0006\n\u0006\n\u0006\n\u0006\n\u0006\n\u0006\n\u0006\n\u0006\n\u0006\n\u0006\n\u0007\n\u0007\n\u0007\n\u0007\n\u0007\n\u0007\n\u0007\n\u0007\n\u0007\n\u0007\n\u0007\n\u0007\n\u0007\n\u0007\n\u0007\n\u0007\n\u0007\n\b\n\b\n\b\n\b\n\b\n\b\n\b\n\b\n\b\n\b\n\b\n\b\n\b\n\b\n\b\n\b\n\b\n\t\n\t\n\t\n\t\n\t\n\t\n\t\n\t\n\t\n\t\n\t\n\t\n\t\n\t\n\t\n\t\n\t\n\n\u000b\n\u000b\n\u000b\n\u000b\n\u000b\n\u000b\n\u000b\n\u000b\n\u000b\n\u000b\n\u000b\n\u000b\n\u000b\n\u000b\n\u000b\n\u000b\n\u000b\n\u0011\n\u0012\n\u0013\n\u0014\n\u0015\n\t\n\t\n\u001e\n\u001e\n0\n\n0\n\"\n\u000b\n\u0016\n,\n\u001e\n!\n!\n\u0007\n\f(c)\n\ny\ng\nr\ne\nn\nE\n\ny\ng\nr\ne\nn\nE\n\nVr3\n\nVr2\n\nVr1\n\nMOSFET\n\nData matched (H-H state)\n\nD1\n\nD1\n\nDc\n\n\u0004\u0002\u0004\n\u0005\u0002\u0005\n\u0005\u0002\u0005\n\nPosition of eM\n\nData unmatched (L-H state)\n\n\u0007\u0002\u0007\n\u0006\u0002\u0006\n\u0006\u0002\u0006\n\nPosition of eM\n\n(d)\n\nt 0\n\n400\n\n0\n\n)\n\nV\ne\nm\n\n(\n \ny\ng\nr\ne\nn\nE\n\n-400\n\nDc\n\nPosition of eM \n\nD1\n\n400\n\n0\n\n)\n\nV\ne\nm\n\n(\n \ny\ng\nr\ne\nn\nE\n\n-400\n\nDc\n\nPosition of eM \n\nD1\n\n(a)\n\nVd1\n\nVd3\n\nVd2\n\nAh\n\nAv\n\nCo\n\nC1\n\nD1\n\n(b)\n\nVd2\nVd1\n\nAh\n\nC2\n\n\u0001\u0002\u0001\n\nD5\n\n: Electron e\n\nM\n\nC2\n\nCj : 0.1aF\nC1 : 0.06aF\nC2 : 0.02aF\nC3 (parasitic) : 0.002aF\nCo : 100aF\n\nAv\n\nCo\n\nC3\n\nDc\n\nCj\n\nC1\n\nVr2\n\nVr1\n\nD1\n\nD5\n\nDv3\n\nDv1\n\nNe ~ S\n\ni\n\n |Vdi - Vri|\n\nVbg\n\n)\nc\ne\ns\n(\n \n\nM\n\nt\n \ne\nm\n\ni\nt\n \ng\nn\ni\nv\no\nm\nM\ne\n\n \n\n10-4\n\n10-6\n\n10-8\n\nH-H state\nL-H state\n\n200\n\n300\n\nTemperature (K)\n\nFigure 5: Nanostructure evaluating Hamming distance. (a) Schematic of nanostructure,\nwhere dot arrays are extremely enlarged compared with a MOSFET to emphasize the dot\n\nstructures. (b) Single-electron circuit. (c) Potential pro\ufb01le in dot array2\n\t . (d) \u0002\f\u000b moving\n\ntime for bit comparator operation.\n\n4 Nanostructure implementing SA model\n\nThe key for implementing the SA model is adding random \ufb02uctuation as expressed by\neq. (8). We have already proposed single-electron circuits and nanostructures evaluating\nHamming distance for the SA model [5]-[9].\n\n(\n\n\u000f\u0011-\n\n\u000f\u001c-\n\n(\n\n-\u0016\n\n\u000f\u0012\u0011\u0013\u0011\u0012\u0011&\u000f\u001c-\u001a\u001d\n\n\u000f\u0011-\n\u000f\u0012\u0011 ), where & means the number of dots at a side of 2\n\nFigure 5(a) and (b) show a nanostructure and the corresponding single-electron circuit, re-\nspectively, which are the most sophisticated version of our circuits and structures [9]. The\n) dot structures arranged on a MOS transistor gate elec-\nnanostructure consists of plural (\r\ntrode. Each dot structure consists of 1-D dot arrays2\u000e\t\n\r ) and\n\u000f\u0012\u0011\u0012\u0011\u0013\u0011\u0012\u000f\u001c-\n\u000f\u0011-\u001a\u001d\n2\u0010\u000f\n\t . (From Monte\nshould be more than 3). The dot diameter as-\nCarlo single-electron circuit simulation, &\nsumed is around 1 nm. The capacitance \u0013\u0015\u0014 corresponds to the gate capacitance of an\nis introduced in array2\u000e\t , which is for example\nultrasmall MOS transistor. An electron \u0002\u0016\u000b\n\u001c . Electron\nperformed by using Fowler-Nordheim tunneling from the grounded plate over\n\u001c , can move along array2\n% . Digital (High/Low) volt-\n, but it cannot move to 2\nthrough the normal capacitor \u0013\nages \u0018\u0017\n) are applied at both edges of 2\n\t , which correspond to\n\u000f\u0019\r\n\u000f\u0012\u0011\u0013\u0011\u0012\u0011\nor \u001b ) are matched, electron \u0002\f\u000b\n\u001c ,\nIf the two digital data bits (\u001a\notherwise \u0002\f\u000b moves to an off-center position. After stabilizing \u0002\f\u000b\n\nelements of data and reference vectors, respectively. Each dot structure simply works as an\nexclusive-NOR logic gate (bit comparator) with random \ufb02uctuation as explained below.\n\n, which is initially located at\n\nthrough tunneling junctions\n\n, by changing voltages\n\nstabilizes at center dot\n\n0 and \n\n0 (\n\n\n\u0003\n\b\n\u001c\n-\n\u000f\n\n\u000f\n%\n-\n\u0002\n\u000b\n-\n\t\n\u0013\n\n\u000f\n5\n\u001a\n\b\n\n\u000f\n\u000f\n-\n\fstays at\n\n\u001c ,2\n\nin array2\n\n\u001c and both of\n\n\u000f detects whether \u0002\f\u000b\n\nstays at\n\nis polarized and an electron is induced at the gate electrode\n\n0 and back-gate voltage\u0001\u0003\u0002 , vertical dot array2\n\u001c or\n\u0014 . The total number of induced electrons (\u0007\u0005\u0004 ) is proportional to the number of dot\n\n\u0018\u0017\n0 ,\nnot; only if \u0002\f\u000b\nof \u0013\nstructures with matched bits; thus the Hamming distance can be measured by counting the\ninduced electrons using the ultrasmall MOS transistor. (If one of the input digital data is\napplied through an inverter, the number of unmatched bits can be calculated).\nThe detail of operation stabilizing \u0002\nis as follows: Because of the charging energy of \u0002\n\t has two peaks at the\nitself, the total energy as a function of the position of \u0002\n\r as shown\nmidpoints of each side of the array, and has minimal values at\nin Fig. 5(c). The energy barrier height for \u0002\f\u000b\nis assumed larger than the thermal\nenergy at room temperature.\n\u001c . On\nIn L-L state, the energy at\nrises up, thus \u0002\nthe other hand, in H-L(L-H) or H-H state, the energy barrier is lower than that of L-L\nstate, thus \u0002\f\u000b\ncan more easily overcome the barrier by using thermal noise. Figure 5(d)\n\u000b moves\nshows the relation between operation temperature and time (8\n) required until \u0002\n\r , which was obtained by Monte Carlo single-electron circuit simulation. The moving\nto\nprocess assisted by thermal noise is purely stochastic, thus 8\nscatters in a wide range.\nHowever, because the energy barrier height in H-L(L-H) states is lower than that in H-H\nstate as shown in Fig. 5(c), there exists a certain time span8\nin H-L(L-H)\n\r while \u0002\f\u000b\nis\nin H-H state stays at\n/ depends on the tunneling resistance. If the\nseveral microseconds in this case although 8\n/ , nearly perfect exclusive-NOR (bit comparison) operation\ndetection process starts after8\n/ , arbitrary amount of\nis achieved. On the other hand, if the start timing is shifted from 8\n\ufb02uctuation can be included in the bit comparison result. Thus, we utilize quantum mechan-\nical tunneling processes assisted by thermal noise in this structure, which is similar to a\nphenomenon known as stochastic resonance.\n\n/ within which \u0002\n\n\u001c . At room temperature (300K),8\n\nat\n\nis most strongly stabilized at\n\nstates moves to\n\nAlthough digital data are treated in the above explanation, analog data can be treated in the\nsame circuit by using pulse-width modulation (PWM) signals, which have a digital am-\nplitude and an analog pulse width [10]. Therefore, instead of the Hamming distance, the\nManhattan distance can be evaluated by using this nanostructure. Because random \ufb02uctu-\nation is naturally added in our nanostructure, it can implement the calculation expressed\nby eq. (8). The annealing procedure described by eqs. (13) and (14) can be performed by\nchanging the time scale in the stabilization operation; that means the scaling of pulse-width\nmodulation.\n\nThe proposed nanostructure has not yet been fabricated using the present VLSI technology,\nbut the basic technology related to nanocrystalline \ufb02oating-dot MOSFET devices, which\nare closely related to our structure, is now being developed [11]-[13]. Furthermore, well-\ncontrolled self-assembly processes using molecular manipulation technology, especially\nusing DNA [14], would be utilized to fabricate our nanostructure. Thus, it could be con-\nstructed in the near future.\n\n5 Conclusions\n\nThe stochastic association algorithm offers a simple and powerful soft-max adaptation rule\nfor vector quantizers. Although it is the same as the simple on-line K-means clustering\nmethod except for adding random \ufb02uctuation in the distortion error evaluation process, our\nnew method has an ef\ufb01cient adaptation performance as high as the neural-gas (NG) or the\nSOM algorithms. Moreover, our method needs no additional process such as sorting and\nonly one reference vector is adjusted at each adaptation step; thus the computational effort\nis much smaller compared with the conventional soft-max clustering algorithms.\n\n5\n-\n-\n\u000f\n\u000b\n\u000b\n\u000b\n-\n-\n-\n\u001c\n-\n\n\u000b\n-\n\u000b\n-\n\u000b\n\u000b\n-\n-\n/\n\fBy employing the nanostructure proposed in this paper, very high performance clustering\nhardware could be constructed.\n\nAcknowledgments\n\nThe authors wish to thank Prof. Masataka Hirose for his support and encouragement. This\nwork has been supported in part by Grants-in-aid for the Core Research for Evolutional\nScience and Technology (CREST) from Japan Science and Technology Corporation(JST).\n\nReferences\n\n[1] K. Rose, E. Gurewitz, and G. C. Fox, \u201cStatistical Mechanics and Phase Transitions in Cluster-\n\ning,\u201d Physical Review Letters, vol. 65, no. 8, pp. 945\u2013948, 1990.\n\n[2] T. Kohonen, Self-Organization and Associative Memory, Springer-Verlag, Berlin, 1984.\n[3] T. M. Martinetz, S. G. Berkovich, and K. J. Schulten, \u201c\u201cNeural-Gas\u201d Network for Vector Quan-\ntization and its Apllication to Time-Series Prediction,\u201d IEEE Trans. Neural Networks, vol. 4,\npp. 558\u2013569, 1993.\n\n[4] S. Rovetta and R. Zunino, \u201cEf\ufb01cient Training of Neural Gas Vector Quantizers with Analog\n\nCircuit Implementation,\u201d IEEE Trans. Circuits & Syst., vol. 46, pp. 688\u2013698, 1999.\n\n[5] M. Saen, T. Morie, M. Nagata, and A. Iwata, \u201cA Stochastic Associative Memory Using Single-\n\nElectron Tunneling Devices,\u201d IEICE Trans. Electron., vol. E81-C, no. 1, pp. 30\u201335, 1998.\n\n[6] T. Yamanaka, T. Morie, M. Nagata, and A. Iwata, \u201cA Single-Electron Stochastic Associa-\ntive Processing Circuit Robust to Random Background-Charge Effects and Its Structure Using\nNanocrystal Floating-Gate Transistors,\u201d Nanotechnology, vol. 11, no. 3, pp. 154\u2013160, 2000.\n\n[7] T. Morie, T. Matsuura, S. Miyata, T. Yamanaka, M. Nagata, and A. Iwata, \u201cQuantum Dot Struc-\ntures Measuring Hamming Distance for Associative Memories,\u201d Superlattices & Microstruc-\ntures, vol. 27, no. 5/6, pp. 613\u2013616, 2000.\n\n[8] T. Matsuura, T. Morie, M. Nagata, and A. Iwata, \u201cA Multi-Quantum-Dot Associative Circuit\nUsing Thermal-Noise Assisted Tunneling,\u201d inExt. Abs. of Int. Conf. on Solid State Devices and\nMaterials, pp. 306\u2013307, Sendai, Japan, Aug. 2000.\n\n[9] T. Morie, T. Matsuura, M. Nagata, and A. Iwata, \u201cQuantum Dot Structures Measuring Hamming\nDistance for Associative Memories,\u201d in Extended Abstracts, 4th International Workshop on\nQuantum Functional Devices (QFD2000), pp. 210\u2013213, Kanazawa, Japan, Nov. 2000.\n\n[10] A. Iwata and M. Nagata, \u201cA Concept of Analog-Digital Merged Circuit Architecture for Future\n\nVLSI\u2019s,\u201d IEICE Trans. Fundamentals., vol. E79-A, no. 2, pp. 145\u2013157, 1996.\n\n[11] S. Tiwari, F. Rana, H. Hana\ufb01,A. Hartstein, E. F. Crabb\u00b4e, and K. Chan, \u201cA Silicon Nanocrystals\n\nBased Memory,\u201d Appl. Phys. Lett., vol. 68, no. 10, pp. 1377\u20131379, 1996.\n\n[12] A. Kohno, H. Murakami, M. Ikeda, H. Nishiyama, S. Miyazaki, and M. Hirose, \u201cTransient\nCharacteristics of Electron Charging in Si-Quantum-Dot Floating Gate MOS Memories,\u201d in\nExt. Abs. of Int. Conf. on Solid State Devices and Materials, pp. 124\u2013125, Sendai, Japan, Aug.\n2000.\n\n[13] R. Ohba, N. Sugiyama, J. Koga, K. Uchida, and A. Toriumu, \u201cNovel Si Quantum Memory\nStructure with Self-Alighed Stacked Nanocrystalline Dots,\u201d in Ext. Abs. of Int. Conf. on Solid\nState Devices and Materials, pp. 122\u2013123, Sendai, Japan, Aug. 2000.\n\n[14] R. A. Kiehl, \u201cNanoelectronic Array Architecture,\u201d inExtended Abstracts, 4th International\nWorkshop on Quantum Functional Devices (QFD2000), pp. 49\u201351, Kanazawa, Japan, Nov.\n2000.\n\n\f", "award": [], "sourceid": 1947, "authors": [{"given_name": "Takashi", "family_name": "Morie", "institution": null}, {"given_name": "Tomohiro", "family_name": "Matsuura", "institution": null}, {"given_name": "Makoto", "family_name": "Nagata", "institution": null}, {"given_name": "Atsushi", "family_name": "Iwata", "institution": null}]}