{"title": "Information Theoretic Analysis of Connection Structure from Spike Trains", "book": "Advances in Neural Information Processing Systems", "page_first": 515, "page_last": 522, "abstract": null, "full_text": "Information Theoretic Analysis of \n\nConnection Structure from Spike Trains \n\nSatoru Shiono\u00b7 \n\nCen tral Research Laboratory \n\nMi tsu bishi Electric Corporation \nAmagasaki, Hyogo 661, Japan \n\nSatoshi Yamada \n\nCentral Research Laboratory \n\nMitsu bishi Electric Corporation \nAmagasaki, Hyogo 661, Japan \n\nMichio Nakashima \n\nCen tral Research Laboratory \n\nMi tsu bishi Electric Corporation \nAmagasaki, Hyogo 661, Japan \n\nKenji Matsumoto \n\nFacul ty of Pharmaceu tical Science \n\nHokkaidou University \n\nSapporo, Hokkaidou 060, Japan \n\nAbstract \n\nWe have attempted to use information theoretic quantities for ana(cid:173)\nlyzing neuronal connection structure from spike trains. Two point \nmu tual information and its maximum value, channel capacity, be(cid:173)\ntween a pair of neurons were found to be useful for sensitive de(cid:173)\ntection of crosscorrelation and for estimation of synaptic strength, \nrespectively. Three point mutual information among three neurons \ncould give their interconnection structure. Therefore, our informa(cid:173)\ntion theoretic analysis was shown to be a very powerful technique \nfor deducing neuronal connection structure. Some concrete exam(cid:173)\nples of its application to simulated spike trains are presented. \n\n1 \n\nINTRODUCTION \n\nThe deduction of neuronal connection structure from spike trains, including synaptic \nstrength estimation, has long been one of the central issues for understanding the \nstructure and function of the neuronal circuit and thus the information processing \n\n\u00b7corresponding author \n\n515 \n\n\f516 \n\nShiono, Yamada, Nakashima, and Matsumoto \n\nmechanism at the neuronal circuitry level. A variety of crosscorrelational techniques \nfor two or more neurons have been proposed and utilized (e.g., Melssen and Epping, \n1987; Aertsen et. ai., 1989). There are, however, some difficulties with those \ntechniques, as discussed by, e.g., Yang and Shamma (1990). It is sometimes difficult \nfor the method to distinguish a significant crosscorrelation from noise, especially \nwhen the amount of experimental data is limited. The quantitative estimation \nof synaptic connectivity is another difficulty. And it is impossible to determine \nwhether two neurons are directly connected or not, only by finding a significant \ncrosscorrelation between them. \n\nThe information theory has been shown to afford a powerful tool for the description \nof neuronal input-output relations, such as in the investigation on the neuronal cod(cid:173)\ning of the visual cortex (Eckhorn et. ai., 1976; Optican and Richmond, 1987). But \nthere has been no extensive study to apply it to the correlational analysis of action \npotential trains. Because a correlational method using information theoretic quan(cid:173)\ntities is considered to give a better correlational measure, the information theory is \nexpected to offer a unique correlational method to overcome the above difficulties. \n\nIn this paper, we describe information theory-based correlational analysis for action \npotential trains, using two and three point mutual information (MI) and channel \ncapacity. Because the information theoretic analysis by two point MI and channel \ncapacity will be published in near future (Yamada et. ai., 1993a), more detailed de(cid:173)\nscription is given here on the analysis by three point MI for infering the relationship \namong three neurons. \n\n2 CORRELATIONAL ANALYSIS BASED ON \n\nINFORMATION THEORY \n\n2.1 \n\nINFORMATION THEORETIC QUANTITIES \n\nAccording to the information theory, the n point mutual information expresses the \namount of information shared among n processes (McGill, 1955). Let X, Y and \nZ be processes, and t and s be the time delays of X and Y from Z, respectively. \nUsing Shannon entropies H, two point MI between X and Y and three point MI, \nare defined (Shannon, 1948; Ikeda et. ai., 1989): \n\nI(Xt : Ys ) \nI(Xt : Y, : Z) \n\nH(Xt ) + H(Y,) - H(Xt, Y,), \nH(Xt ) + H(Y,) + H(Z) - H(Xt, y,) \n-H(Y\" Z) - H(Z, X t ) + H(Xt, Y\" Z). \n\n(1 ) \n\n(2) \n\nI(Xt : Ys : Z) is related to I(Xt : Y,) as follows: \n\n(3) \nwhere I(Xt : YsIZ) means the two point conditional MI between X and Y if the \nstate of Z is given. On the other hand, channel capacity is given by (r = s - t), \n\nCC(X: Yr) = maxI(X: Yr). \n\np(x,) \n\n( 4) \n\nWe consider now X, Y and Z to be neurons whose spike activity has been measured. \n\n\fInformation Theoretic Analysis of Connection Structure from Spike Trains \n\n517 \n\nTwo point MI and two point conditional MI are obtained by (i, j, k = 0, 1), \n\n(X Y) ~ ( I) ( )1 \nI \n\n: T = L....J P Yj,T Xi P Xi og ( . ) ' \n\np(Yj,Tlxi) \n\nP YJ,T \n\n. . \nI,J \n\nI(X YIZ) ~( I ) ( ) l \n\nt : , = L....J P Xi,t, Yj\" Zk P Zk og (x. Iz ) ( . Iz ). \n\np(xi,t,Yj\"lzk) \nI,t k P YJ,s \nk \n\nP \n\n. . \u2022. \nI,J,'\" \n\n(5) \n\n(6) \n\nwhere x, Y and z mean the states of neurons, e.g., Xl for the firing state and \nXo for the non-firing state of X, and p( ) denotes probability. And three point \nMI is obtained by using Equation (3). Those information theoretic quantities are \ncalculated by using the probabilities estimated from the spike trains of X, Y and Z \nafter the spike trains are converted into time sequences consisting of 0 and 1 with \ndiscrete time steps, as described elswhere (Yamada et. al., 1993a). \n\n2.2 PROCEDURE FOR THREE POINT MUTUAL INFORMATION \n\nANALYSIS \n\nSuppose that a three point MI peak is found at (to, so) in the t, s-plane (see Figure 1). \nThe three time delays, to, So and r = So - to, are obtained. They are supposed to be \ntime delays in three possible interconnections between any pair of neurons. Because \nthe peak is not significant if only one pair of the three neurons is interconnected, two \nor three of the possible interconnections with corresponding time delays should truly \nwork to produce the peak. We will utilize I(n : m) and I(n : mil) (n, m, I = X, Y or \nZ) at the peak to find working interconnections out of them. These quantities are \nobtained by recalculating each probability in Equations (5) and (6) over the whole \npeak region. \nIf two neurons, e.g., X and Y, are not interconnected either I(X : Y) or I(X : YIZ) \nis equal to zero. The reverse proposition, however, is not true. The necessary \nand sufficient condition for having no interconnection is obtained by calculating \nI( n : m) and I( n : mil) for all possible interconnection structures. The neurons are \nrearranged and renamed A, Band C in the order of the time delays. There are only \nfour interconnection structures, as shown in Table 1. \n\nI: No interconnection between A and B. A and B are statistically independent, i. e., \np(aj,bj ) = p(aj)p(bj ), I(A: B) = O. The three point MI peak is negative. \nII: No interconnection between A and C. The states of A and C are statistically in(cid:173)\ndependent when the state of B is given, i.e., p(ai' cklbj) = p(adbj)p(Cklbj), \nI(A : CIB) = O. The peak is positive. \nIII: No interconnection between Band C. Similar to case II, because p(bj , cklai) = \np(bjlai)p(cklai), I(B: CIA) = O. The peak is positive. \n\nIV: Three in terconnections. The above three cases are considered to occur concomi(cid:173)\n\ntantly in this case. The peak is positive or negative, depending on their \nrelative contributions. Because A and B should have an apparent effect on \nthe firing-probability of the postsynaptic neurons, I(A : B), I(A : CIB) \nand I(B : CIA) are all non-zero except for the case where the activity of \nB completely coincides with that of A with the specified time delay (in \nthis case, both I(A : CIB) and I(B : CIA) are zero (see Yamada et. al., \n1993b)). \n\n\f518 \n\nShiono, Yamada, Nakashima, and Matsumoto \n\nTable 1. Interconnection Structure and Information Theoretic Quantities \n\nInterconnection \n\nStructure \n\n2 point MI \n\nI(A:B) \nI(A:C) \nI(B:C) \n\n2 point condition MI \n\nI(A:B I C) \nI(A:CIB) \nI(B:C I A) \n\n3 point MI \n\nI(A:B:C) \n\nI: ~ II: ~ III: @ \n~@cl@ \n\n~@ \n\nIV: ~ \n\nctJ@ \n\n=0 \n~O \n~O \n\n>0 \n>0 \n>0 \n\n>0 \n>0 \n>0 \n\n~O \n=0 \n~O \n\n+ \n\n>0 \n>0 \n>0 \n\n~O \n~O \n=0 \n+ \n\n>0 \n~O \n~O \n\n~O \n>0 \n>0 \n+ or -\n\nFrom what we have described above, the interconnection structure for a three point \nMI peak is deduced utilizing the following procedure; \n\n(a) A negative 3pMI peak: it corresponds to case I or IV. The problem is to \n\ndetermine whether A and B are interconnected or not. \n(1) If I(A : B) = 0, case I. \n(2) If I(A : B) > 0, case IV. \n\n(b) A positive 3pMI peak: it corresponds to case II, III or IV. The existence of the \n\nA-C and B-C interconnections has to be checked. \n\n(1) If I(A : CIB) > \u00b0 and I(B : CIA) > 0, case IV. \n(2) If I(A : CIB) = \u00b0 and I(B : CIA) > 0, case II. \n(3) If I(A : CIB) > \u00b0 and I(B : CIA) = 0, case III. \n(4) If I(A : CIB) = \u00b0 and I(B : CIA) = 0, the interconnection structure \n\ncannot be ded nced except for the A - B interconnection. \n\nThis procedure is applicable, if all the time delays are non-zero. If otherwise, some \nof the interconnections cannot be determined (Yamada et. ai., 1993b). \n\n3 SIMULATED SPIKE TRAINS \n\nIn order to characterize our information theoretic analysis, simulations of neu(cid:173)\nronal network models were carried out. We used a model neuron described by \n\n\fInformation Theoretic Analysis of Connection Structure from Spike Trains \n\n519 \n\nthe Hodgkin-Huxley equations (Yamada et. ai., 1989). The used equations and pa(cid:173)\nrameters were described (Yamada et. al., 1993a). The Hodgkin-Huxley equations \nwere mathematically integrated by the Runge-Kutta-Gill technique. \n\n4 RESULTS AND DISCUSSION \n\n4.1 ANALYSIS BY TWO POINT MUTUAL INFORMATION AND \n\nCHANNEL CAPACITY \n\nThe performance was previously reported of the information theoretic analysis by \ntwo point MI and channel capacity (Yamada et. ai., 1993a). \n\nBriefly, this anlytical method was compared with some conventional ones for both \nexcitatory and inhibitory connections using action potential trains obtained by the \nsimulation of a model neuronal network. It was shown to have the following ad(cid:173)\nvantages. First, it reduced correlational measures within the bounds of noise and \nsimultaneously amplified beyond the bounds by its nonlinear function. It should be \neasier in its crosscorrelation graph to find a neuron pair having a weak but signif(cid:173)\nicant interaction, especially when the synaptic strength is small or the amount of \nexperimental data is limited. Second, channel capacity was shown to allow fairly \neffective estimation of synaptic strength, being independent of the firing probability \nof a presynaptic neuron, as long as this firing probability was not large enough to \nhave the overlap of two successive postsynaptic potentials. \n\n4.2 ANALYSIS BY THREE POINT MUTUAL INFORMATION \n\nThe practical application of the analysis by three point MI is shown below in detail, \nusing spike trains obtained by simulation of the three-neuron network models shown \nin Figures 1 and 2 (Yamada et. ai., 1993b). \n\nThe network model in Figure 1(1) has three interconnections. In Figure 1(2), three \npoint MI has two positive peaks at (17ms, 12ms) (unit \"ms\" is omitted hereafter) \nand (17,30), and one negative peak at (0,12). For the peak at (17,12), the neurons \nare renamed A, B and C from the time delays (Z as A, Y as B and X as C), as \nin Table 1. Because only I(B : CIA) ..:. 0 (see Figure 1 legend), the peak indicates \ncase III with A-+B (Z-+Y) (s = 12) and A-+C (Z-+X) (t = 17) interconnections. \nSimilarly, the peak at (17,30) indicates Z -+X and X -+Y (s - t = 13) intercon(cid:173)\nnections, and the peak at (0,12) indicates Z-+Y and X -+Y interconnections. The \ninterconnection structure deduced from each three point MI peak is consistent with \neach other, and in agreement with the network model. \n\nAlternatively, the three point MI graphical presentation such as shown in Figure \n1(2) itself gives indication of some truly existing interconnections. If more than two \nthree point MI peaks are found on one of the three lines, t = to, s = So and s-t = TO, \nthe interconnection with the time delay represented by this line is considered to be \nreal. For example, because the peaks at (17, 12) and (17, 30) are on the line of t = 17 \n(Figure 1(2)), the interconnection represented by t = 17 (Z-+X) are considered to \nt = 12 \nbe real. In a similar manner, the interconnections of s = 12 (Z-+Y) and s -\n(X -+Y) are obtained. But this graphical indication is not complete, and thus the \ncalculation of two point MI's and two point conditional MI's should be always \n\n\f520 \n\n5hiono\u2022 Yamada. Nakashima. and Matsumoto \n\n(1) \n\n(2) \n\n0.0010 \n\n-so \n\n~.oo10 \n\nNeuron X ~ Neuron Y \n\nDNeuronZ \n\no \n\nt (ms) \n\nso \n\n00\n\nFigure 1. Three point Ml analYsis of simulated spike trains. (1) A. three-neuron \nnetwork model with Z .... X Z .... Y and X .... Y interconnections. The total number of \n\u2022 y:5400\u2022 Z:3150. (2) Three poinl Ml analysis of spike trains. Three \nspikes; X:40\npoint Ml has two positive peaks al (17.12) and (17.30). and one negative peak at \n(0.12). For the peak al (17. 12) the neurons are renamed (Z as A. Y as B and X \nas C). Two point Ml and two point conditional M1 for the peak at (17. 12) are: \nI(A: B) == 0.03596. I(A: C) == 0.06855\u2022 I(B : C) == 0.01375\u2022 I(A : BIC) == 0.02126. \nI(A : CIB) == 0.05376. I(B : CIA) == 0.00011. So. I( B : CIA) .:. o. indicating case \n111 (see Table 1) with A .... B (Z .... Y) and A .... C (Z .... X) interconnections. Similarly, \nfor the peaks at (17,30) and at (0,12). Z .... X and X .... Y interConnections. and Z .... Y \nand X .... Y interconnections are obtalned, respectively. \n\nperformed for connrma.tion. \nThe nelwork model in Figure 2(1) has four interCOnnections. Three point Ml has \n12\nftVe major peaks: four positive peaks at (17. -12). (17. 30). (_24.-\n) \nand one negative peak at (0.10). The peaks at (17. -12). (17. 12) and (17. 30) Me \non the line 01 t == 17 (Z .... X). the peaks at (17, -12) and (-24. -12) are on Il\\e \nline 01 s == -12 (Z ... Y). the peaks at (17.12) and (0. 10) are on the line of s == 12 \n(Z .... Y). and the peaks at (-24. -12), (0.10) and (17. 30) are on the line of \n\n) and (1\n\n12\n\n7\n\n\u2022\n\n\fInformation Theoretic Analysis of Connection Structure from Spike Trains \n\n521 \n\n(1) \n\n(2) \n\n0.0008 \n\n-0.0008 \n\nNeuron X ~ Neuron Y \n\n~euronz \n\no \n\nt (ms) \n\n50 \n\nFigure 2. Three point MI analysis of simulated spike trains. (1) A three-neuron \nnetwork model with Z-+X Z-+Y, Z~Y and X-+Y interconnections. The total \nnumber of spikes; X:4300, Y:5150, Z:4850. (2) Three point MI analysis of spike \ntrains. Three point MI has five major peaks, four positive peaks at (17, -12), \n(17,12), (17,30) and (-24, -12), and one negative peak at (0,10). \n\ns - t = 12 (X -+Y). The calculation of two point MI and two point conditional MI \nfor each peak gives the confirmation that each three point MI peak was produced \nby two interconnections. Namely, their calculation indicates Z-+X (t = 17), Z~Y \n(s = -12), Z-+Y (s = 12) and X-+Y (s - t = 12) interconnections. There are \nalso some small peaks. They are considered to be ghost peaks due to two or three \ninterconnections, at least one of wllich is a combination of two interconnections \nfound by analyzing the major peaks. For example, the positive peak at (-7, -12) \nindicates Z~Y and X-+Y interconnections, but the latter (s - t = -5) is the \ncombination of the Z -+ X interconnection (t = 17) and the Z -+ Y interconnection \n(s = 12). \nThe interconnection structure of a network containing an inhibitory intercolLllectioll \nor consisting of more than four neurons can also be deduced, although it becomes \nmore difficult to perform the three point MI analysis. \n\n\f522 \n\nShiono, Yamada, Nakashima, and Matsumoto \n\nReferences \n\nA. M. H. J. Aertsen, G. L. Gerstein, M. K. Habib & G. Palm. (1989) Dynamics of \nneuronal firing correlation: modulation of\" effective connectivity\". J. N europhY6iol. \n61: 900-917. \nR. Eckhorn, O. J. Griisser, J. Kremer, K. Pellnitz & B. Popel. (1976) Efficiency \nof different neuronal codes: \ninformation transfer calculations for three different \nneuronal systems. Bioi. Cyhern. 22: 49-60. \n\nK. Ikeda, K. Otsuka & K. Matsumoto. (1989) Maxwell-Bloch turbulence. Prog. \nTheor. Phys., Suppl. 99: 295-324. \nW. J. McGill. \nTheory 1: 93-111. \n\n(1955) Multivariate information transmission. \n\nIRE Tran6. \n\nInf. \n\nW. J. Melssen & W. J. M. Epping. (1987) Detection and estimation of neural \nconnectivity based on crosscorrelation analysis. Bioi. Cyhern. 57: 403-414. \n\nL. M. Optican & B. J. Richmond. (1987) Temporal encoding of two-dimensional \npatterns by single units primate inferior temporal cortex. III. Information theoretic \nanalysis. J. NeurophY6iol. 57: 162-178. \n\nC. E. Shannon. \nTechn. J. 27: 379-423. \n\n(1948) A mathematical theory of communication. Bell. Syst. \n\nS. Yamada, M. Nakashima, K. Matsumoto & S. Shiono. (1993a) Information the(cid:173)\noretic analysis of action potential trains: 1. Analysis of correlation between two \nneurons. Bioi. Cyhern., in press. \n\nS. Yamada, M. Nakashima, K. Matsumoto & S. Shiono. (1993b) Information the(cid:173)\noretic analysis of action potential trains: II. Analysis of correlation among three \nneurons. submitted to BioI. Cyhern. \n\nW. M. Yamada, C. Koch & P. R. Adams. (1989) Multiple channels and calcium \ndynamics. \nIn C. Koch & I. Segev (ed), Methods in Neuronal Modeling: From \nSynapses to Neurons, 97-133, Cambridge, MA, USA: MIT Press. \n\nX. Yang & S. A. Shamma. (1990) Identification of connectivity in neural networks. \nBiophys. J. 57: 987-999. \n\n\f", "award": [], "sourceid": 680, "authors": [{"given_name": "Satoru", "family_name": "Shiono", "institution": null}, {"given_name": "Satoshi", "family_name": "Yamada", "institution": null}, {"given_name": "Michio", "family_name": "Nakashima", "institution": null}, {"given_name": "Kenji", "family_name": "Matsumoto", "institution": null}]}