{"title": "Correlation Codes in Neuronal Populations", "book": "Advances in Neural Information Processing Systems", "page_first": 277, "page_last": 284, "abstract": null, "full_text": "Correlation Codes in Neuronal Populations\n\nMaoz Shamir and Haim Sompolinsky\n\nRacah Institute of Physics and Center for Neural Computation,\nThe Hebrew University of Jerusalem, Jerusalem 91904, Israel\n\n\u0002\u0001\u0004\u0003\u0006\u0005\b\u0007\n\t\f\u000b\n\u0003\u0006\r\u000e\u0001\u0010\u000f\u0012\u0011\u0013\r\u0014\u0007\n\u0015\n\n\u000b\u0017\u0016\u0019\u0018\u001a\r\u001b\u0015\n\n\u0003\u0006\u001c\u001d\u0015\n\n\u0014\u001e\u0014\u001f\n\nAbstract\n\nPopulation codes often rely on the tuning of the mean responses to the\nstimulus parameters. However, this information can be greatly sup-\npressed by long range correlations. Here we study the ef\ufb01ciency of cod-\ning information in the second order statistics of the population responses.\nWe show that the Fisher Information of this system grows linearly with\nthe size of the system. We propose a bilinear readout model for extract-\ning information from correlation codes, and evaluate its performance in\ndiscrimination and estimation tasks. It is shown that the main source of\ninformation in this system is the stimulus dependence of the variances of\nthe single neuron responses.\n\n1 Introduction\n\nExperiments in the last years have shown that in many cortical areas, the \ufb02uctuations in\nthe responses of neurons to external stimuli are signi\ufb01cantly correlated [1, 2, 3, 4], rais-\ning important questions regarding the computational implications of neuronal correlations.\nRecent theoretical studies have addressed the issue of how neuronal correlations affect the\nef\ufb01ciency of population coding [4, 5, 6]. It is often assumed that the information about\nstimuli is coded mainly in the mean neuronal responses, e.g., in the tuning of the mean\n\ufb01ring rates, and that by averaging the tuned responses across large populations, an accu-\nrate estimate can be obtained despite the signi\ufb01cant noise in the single neuron responses.\nIndeed, for uncorrelated neurons the Fisher Information of the population is extensive [7];\nnamely, it increases linearly with the number of neurons in the population. Furthermore, it\nhas been shown that this extensive information can be extracted by relatively simple linear\nreadout mechanisms [7, 8]. However, it was recently shown [6] that positive correlations\nwhich vary smoothly with space may drastically suppress the information in the mean re-\nsponses. In particular, the Fisher Information of the system saturates to a \ufb01nite value as\nthe system size grows. This raises questions about the computational utility of neuronal\npopulation codes.\n\nNeuronal population responses can represent information in the higher order statistics of\nthe responses [3], not only in their means. In this work, we study the accuracy of cod-\ning information in the second order statistics. We call such schemes correlation codes.\nSpeci\ufb01cally, we assume that the neuronal responses obey multivariate Gaussian statistics\ngoverned by a stimulus-dependent correlation matrix. We ask whether the Fisher Informa-\ntion of such a system is extensive even in the presence of strong correlations in the neuronal\n\n\fnoise. Secondly, we inquire how information in the second order statistics can be ef\ufb01ciently\nextracted.\n\n2 Fisher Information of a Correlation Code\n\n')(\n\n\u0001\u001c\u001b\u001e\u001d\n\n=6\n\n=6\n\u0001\u001c\u001b\n\n\u0016K>\n\nHere \u0011\n\n\u0001\u001c\u001b\n\n(1)\n\n(2)\n\n(3)\n\n(4)\n\n(5)\n\n(6)\n\nGaussian distribution\n\n!B#\u000e%DC\n\n. Their\nis the activity of the\n\nis usually referred\nis a normalization\nconstant. Here we shall limit ourselves to the case of multiplicative modulation of the\ncorrelations. Speci\ufb01cally we use\n\nOur model consists of a system of\nneurons that code a 2D angle\u0001 ,\u0002\u0004\u0003\u0005\u0001\u0006\u0003\b\u0007\n\t\nstochastic response is given by a vector of activities \f\u000b\u000e\r\u0014\u001f\u0010\u000f\n\r\u0012\u0011\u0014\u0013 where\u000b\u0010\n\n -th neuron in the presence of a stimulus\u0001 , and is distributed according to a multivariate\n(3*\n(+*\n\u0016\u0019\u0018\n\u0016\u0019\u0018\n\u0015\u0017\u0016\u0019\u0018\u0017\u001a\n \"!$#&%\n\u0001\u001c\u001b21\n\u0001\u001c\u001b,\u001b.-0/\n\u0001\u001c\u001b0\u001b\u00104\nis the mean activity of the \r -th neuron and its dependence on\u0001\n\u0001\u001c\u001b is the correlation matrix; and \nto as the tuning curve of the neuron;/\n\r=6\n\r76\n\u0001\u001c\u001b09\n\u0001\u001c\u001b8\u001d\n\u0001:\u001b<;\n\r76)A\n\r76\n\u0016?>\n\u001b\u001e\u001d\u0005@\n\u0016?>\nCGHBI\u001cJ\n!$#&%\n\u0001\u001c\u001b8\u001d\n\u0001:\u001bG\u001d\nwhere \u001c andE are the correlation strength and correlation length respectively;L de\ufb01nes the\n\r denotes the angle at which the variance of the \r -th\ntuning width of the correlations; and>\n\u0001\u001c\u001b , is maximal. An example is shown in Fig. 1. It is important to note that the\nneuron,9\n\r76 which is larger than the contribution of the smooth part\nvariance adds a contribution to5\n\r76\n5PO\n\r76\n\u0001:\u001b\n\u0001:\u001bG\u001d\n\r=6 denotes the smooth part of the correlation matrix and5\nwhere5\n5PQ\n\u0001\u001c\u001bG\u001d\n\nA useful measure of the accuracy of a population code is the Fisher Information (FI). In\nthe case of uncorrelated populations it is well known that FI increases linearly with sys-\ntem size [7], indicating that the accuracy of the population coding improves as the system\nsize is increased. Furthermore, it has been shown that relatively simple, linear schemes\ncan provide reasonable readout models for extracting the information in uncorrelated pop-\nulations [8]. In the case of a correlated multivariate Gaussian distribution, FI is given as\n\nof the correlations. For reasons that will become clear below, we write,\n\ndiagonal part, which in the example of Eqs. (2)-(4) is\n\nLNM\n5PQ\n\u001b,9\n\n\u0001\u001c\u001b.\\\n\u000b\u000e^\n\nof these terms reveals that in general the correlations play two roles. First they control the\n\nR:XZY0[0[ , where\nR\u001cSUT.V2W\nR\u001cSUT.V2W\n\u0001\u001c\u001b,\\\nR:XZY0[0[\n\u0001\u001c\u001b0\u001b.\\\u0012_\n\u0007N]\n\\ denote derivatives of* and/ with respect to\u0001 , respectively. The form\nwhere*\n\\ and/\nef\ufb01ciency of the information encoded in the mean activities*\n\u0001:\u001b (note the dependence of\nSUT.V2W on5\n\u0001\u001c\u001b provides an additional source of information about the stim-\n). Secondly,/\n\u0005\u0010bdc\fe , it\nulus (R&XZY0[0[ ). When the correlations are independent of the stimulus, \r\u001b\u0015\nwas shown [6] that positive correlations, \u001cgf\n\u001b ,\n\u001dih\n\n\u0001:\u001b\n\u0001\u001c\u001b\n\u0001\u001c\u001ba\u001d\n\u0002 , with long correlation length,E\n\n`\u0006\u0015\n\n(8)\n\n(7)\n\n\u0001\u001c\u001b.@\n\u0001\u001c\u001b\n\nthe discontinuous\n\n\u001f\n\u001f\n\u0007\n\u0016\n\u0016\n\u0013\n\u0016\n\n\u0016\n\u0016\n5\n\u0016\n9\n\n\u0016\n6\n\u0016\n5\n;\n5\n\u001d\n;\n5\n\n(\n>\n6\n\u001c\n\u0016\n\u001f\n(\n@\n\u001b\n(\n\u001a\n>\n\n(\n>\n6\n\u001a\nE\nF\n9\n\n\u0016\n9\n\n(\n\n(\nF\nM\n\n\u0016\n5\n\u0016\n\u0016\nA\n\n\u0016\nO\nQ\n\n\u0016\n\u0016\n\u001f\n(\n\u001c\nM\n\n\u0016\n\u0015\nR\n\u001d\nA\n\u001d\n*\n\u0016\n-\n/\n\u0016\n1\n\u0013\n*\n\u0016\n\u001d\n\u001f\n/\n\u0016\n1\n\u0013\n\u0016\n/\n\u0016\nM\n\u0016\nR\n\u0016\n9\n\n\u0016\n\u001c\n\u0016\n\u001f\n\ff =0o\n\nC(f ,y\n\n)\n\nf =\u221260o\n\nf =60o\n\nf =\u2212120o\n\nf =120o\n\n\u2212180\n\n\u2212120\n\n\u221260\n\n0\n\n60\n [deg]\n\n120\n\n180\n\n\u0016K>\n\n\t\u0001\n\nwhere\n\n, Eq. (8), we \ufb01nd it useful to write\n\n\u0002 ,E\n\n\u001f and\n\n\u0001 . Here, \u001c\n\n\u001di\u0002\n\n\u001b , where>\n\nFigure 1: The stimulus-dependent correlation matrix, Eqs. (2)-(4), depicted as a function\n\ndependence ofRNXZY,[\n\n. This implies that in the presence\nof such correlations, population averaging cannot overcome the noise even in large net-\nworks. This analysis however, [6], did not take into account stimulus-dependent correla-\ntions, which is the topic of the present work.\n\nof two angles,5\n\u0001 and\n\t\u0004\u0003\n\u0007 .\ncause the saturation of FI to a \ufb01nite limit at large\nAnalyzing the\nXZY0[0[\n\u0001\u001c\u001b,\u001b\n\u0011\u0014\u0013\n\u0001\u001c\u001b\nis FI of an uncorrelated population with stimulus-dependent variance which equals5\n;R\nR:XZY0[0[\nQ . Evaluating these terms for the multiplicative\nand scales linearly with\nmodel, Eq. (2), we \ufb01nd thatR\nis positive, so thatR\u0007\u0006\nQ . Furthermore, numerical evalua-\ntion of this term shows thatR\nO saturates at large\n\u0016?>\nM\u0001\f\nR:XZY0[\n[\t\b\n\u0016?>\n\u0007\n\u000b\n\n\u0007\n\t\nas shown in Fig. 2. We thus conclude thatR\nXZY0[0[\nincreases linearly with\n, to the FI of variance coding namely toR of an independent population in which\nlarge\n\nSince in our system the information is encoded in the second order statistics of the popula-\ntion responses, it is obvious that linear readouts are inadequate. This raises the question of\nwhether there are relatively simple nonlinear readout models for such systems. In the next\nsections we will study bilinear readouts and show that they are useful models for extracting\ninformation from correlation codes.\n\ninformation is encoded in their activity variances.\n\nto a small \ufb01nite value, so that for large\n\n(9)\n\n(10)\n\n,\n\n(11)\n\nand is equal, for\n\n3 A Bilinear Readout for Discrimination Tasks\n\n\u0013\u0012\u0011\n\n\u0018\u0010\u000f\n\n\u0018\u0010\u000f\n\nIn a two-interval discrimination task the system is given two sets of neuronal activities\n\nlus generated which activity. The Maximum-Likelihood (ML) discrimination yields the\n\n\u0011 generated by two proximal stimuli\u0001 and\u0001\n\n@\u0010\u0001 and must infer which stimu-\n\ny\n\u001d\n>\n\n(\n\u001d\n>\n\n(\n\u0015\n\u001d\nL\n\u001d\n[\nR\n\u001d\nR\nQ\nA\nR\nO\n\t\nR\nQ\n\u001d\n\u001f\n\u0007\n\u000f\n\u0005\n\nC\n\u0016\n5\nQ\n\n\u0016\n\\\n5\nQ\n\n\u0016\nF\nM\nQ\n\nO\n\u001d\n(\nR\nO\nR\n\nR\nQ\n\u001d\n\n\u000e\n>\nC\n9\n\\\n\u001b\n9\n\u001b\nF\nM\n\t\nM\nA\n\f1\n\n0.8\n\n0.6\n\n0.4\n\n]\n\n2\n\u2212\n\ng\ne\nd\n\n[\n \n \n \n \n\nr\nr\no\nc\n\nJ\n\n0.2\n\nx 10\u22123\n\n1\n\n0.5\n\n]\n\n2\n\u2212\n\ng\ne\nd\n\n[\n \n \n \n \n\ns\n\nJ\n\n0\n0\n\n200\n\n400\n\n600\n\n800\n\n1000\n\nN\n\n0\n0\n\n200\n\n400\n\n600\n\n800\n\n1000\n\nN\n\n\f\u000e\n\n\u0002 ,E\n\n\u001d\u0005\u0002\n\ndiscriminability\n\n(4), as a function of the number of neurons in the system. In (b) we show the difference\n- as de\ufb01ned by Eq. (9).\nHere \u001c\n\n(12)\nIt has been previously shown that in the case of uncorrelated populations with mean coding,\nthe optimal linear readouts achieves the Maximum-Likelihood discrimination performance\nin large N [7].\n\nXZY0[0[ , of the stimulus-dependent correlations, Eqs. (2)-\n\t\u0004\u0003\u0001 . Note the different scales in (a) and (b).\nM and the\nM\t\b\u000b\n\n\u0007\u001c\u001b , where \u0002\nR\u001e\u0016\n\u001a\u0010\u000f\n@\u0010\u0001\n\nFigure 2: (a) Fisher Information,R\nbetween the full FI and the contribution of the diagonal term,R\n\u001f andL\n\u0007\n\td\u001b\nprobability of error given by \u0002\n\\ equals\nis coded in the average \ufb01ring rates of the neurons, and take*\n6\u0019\u0018\n\nIn order to isolate the properties of correlation coding we will assume that no information\n\n\u0016\u0005\u0004\n\u0001\u001c\u001b\n\n\u0013\u0007\u0006\n\nrule, the optimal bilinear discriminator (OBD) matrix is given by\n\n=6\u0015\u0014\n\u001b decision refers to\u0001\n\nbilinear readout as a simple generalization of the linear readout to correlation codes. In a\ndiscrimination task the bilinear readout makes a decision according to the sign of\n\n\u001d\u0012\u0011 hereafter. We suggest a\n\r76\u0017\u0016\f\u000b\nwhere aA\n@\u0010\u0001\u001c\u001b . Maximizing the signal-to-noise ratio of this\n\r76\nUsing the optimal weights to evaluate the discrimination error we obtain that in large\nmodel increases linearly with the size of the system, the discriminability increases as \u0003\nSince the correlation matrix/\ndepends on the stimulus,\u0001 , the OBD matrix, Eq. (14), will\n\nthe\nperformance of the OBD saturates the ML performance, Eq. (12). Thus, since FI of this\n.\n\nalso be stimulus dependent. Thus, although the OBD is locally ef\ufb01cient, it cannot be used\nas such as a global ef\ufb01cient readout.\n\n\u0001\u001c\u001b\n\n(13)\n\n(14)\n\n76\n\n4 A Bilinear Readout for Angle Estimation\n\n4.1 Optimal bilinear readout for estimation\n\nTo study the global performance of bilinear readouts we investigate bilinear readouts which\n\u0007 as the estimator\n\nminimize the square error of estimating the angle averaged over the whole range of\u0001 . For\nconvenience we use complex notation for the encoded angle, and write \u001a\n\nO\n\u0015\n\u001d\n\u001d\n\u0016\n\u000e\n\\\n\u0003\n\u0003\n\u001b\n\u001d\n\u0016\n1\n\f\n\u000e\n\u0004\n`\n1\n\u0006\n\u000e\n\u000e\n\\\n\u001d\n\u001a\n\u0015\n\u0013\n\u001d\n\u0005\n\u000f\n\u0013\n\u0011\n\n\u000b\n\u000f\n\u0013\n\u0011\n6\n(\n\u000b\n\u000f\nM\n\u0011\n\n\u000b\n\u000f\nM\n\u0011\n\u0016\n(\n\u0016\n\u0001\nA\n\u0014\n\u001d\n\u0016\n5\n1\n\u0013\n\u0016\n\\\n\u001b\n\u0015\n\n\fof \u0007\n\nwhere\n\n\u0016?>\n\n\u001bG\u001d\u0012\u0010\n\n\u0001\u001c\u001b\u000b\n\n\u0018\u000e\u0016\n\u0001\u001c\u001b\n\n\u0007\r\f\n\u001f\u000e\f\n\n. This form of a readout matrix, Eq. (17),\n\n)\n\n76\n\u0016?>\n\n\u001d\u0011\u0010\n\u001b)\u001d\n\n\u0016K>\n\n\u001d\u0005\n\n\u000b\f\rK\u000b26\n\n76\n\n(15)\n\n(16)\n\n(17)\n\n(18)\n\n(19)\n\nthat minimizes on average the quadratic estimation\n\nerror of an unbiased estimator. This error is given by\n\nmator (OBE) as the set of weights\u0002\n\u001bG\u001d\n\n\u0016\u0004\u0002\n\nwhere\nit is impossible to \ufb01nd a perfectly unbiased estimator for a continuously varied stimulus,\nusing a \ufb01nite number of weights. However, in the case of angle estimation, we can employ\nthe underlying rotational symmetry to generate such an estimator. For this we use the\nsymmetry of the correlation matrix, Eq. (2). In this case one can show that the Lagrange\nmultipliers have the simple form of\nof\n\n\u0001 . Let\n\r=6\n\r76 are stimulus independent complex weights. We de\ufb01ne the optimal bilinear esti-\nM\u0007\u0006\n\t\t\b\n\u0007\n\t\n\u0001\u001c\u001b . In general,\n\u0001\u001c\u001b\nis the Lagrange multiplier of the constraint\n\n\u000f , and the OBE weight matrix is in the form\n\u0001\u001c\u001b)\u001d\n\u0016?>\n!B#\u000e%DC\n\u0016?>\n\td\u001b\n\u001b and\u0010\nwhere\u0010\n\u0016K>\n\u001b can\nguarantees that the estimator will be unbiased. Using these symmetry properties,\u0010\nbe written in the following form (for even\n\u0016?>\n\u0003\"\t\n\u001b\u0014\u0013\n@\u0016\u0015\u0018\u0017\n\u0011\u0014\u0013\nHBI\u001cJ\n\u0016K>\n\u001b . These numerical results (Fig. 3 (a))\nFigure 3 (a) presents an example of the function\u0010\nalso suggest that the function\u0010\npeak at>\n\u0002 . Below we will use this fact to study simpler forms of bilinear readout.\nFurther analysis of the OBE performance in the large\n\u0016?>\nM\u0001\f\n\u0001:\u001b\n\u0016?>\n\u001b0\u001b\nof\nreadout grows linearly with the size of the system,\n\u0016K>\nstudied a bilinear readout of the form of Eqs. (17) and (18) with\u0010\n\u0011 for large systems. Surprisingly we found that for small\u001c and large\n\u0003B\n\r76\n!B#\u000e%DC\neral values of\u001c\n\n\u0011\u0014\u0013\nHBI\u001cJ\n\u0002 . The results of Fig. 4 show that for a given\u001c\n\u0007\n\u0002\u001c\u0002\n\nfunction peak at the origin plus a few harmonics. Restricting the number of harmonics to\nrelatively small integers, we evaluated numerically the optimal values of the coef\ufb01cients\n, these coef-\n\ufb01cients approach a value which is independent of the speci\ufb01cs of the model and equals\n\nFigure 3 (b) shows the numerical calculation of the OBE error (open circles) as a function\n. The dashed line is the asymptotic behavior, given by Eq. (19). The dotted line is\nthe Creamer-Rao bound. From the graph one can see that the estimation ef\ufb01ciency of this\n\nFigure 4 shows the numerical results for the squared average error of this readout for sev-\nthe\n\n=6\n\u0003! and8\u0003\n\n\u001b which has a delta\n\nMotivated by the simple structure of the optimal readout matrix observed in Fig. 3 (a), we\n\nis mainly determined by a few harmonics plus a delta\n\n\u0016K>\n\n_\u0001\u001f\n\n\t\u001a\u0019\n\n\u001b,\u001b\n\n\u0005\nc\n\ntotic result\n\nlimit yields the following asymp-\n\n4.2 Truncated bilinear readout\n\n, but is lower than the bound.\n\n, yielding a bilinear weight matrix of the form\n\n(20)\n\n\u001d\n`\n\u001a\n\u0007\n\u001d\n\u0005\n\u0014\n\u0014\n\u0003\n\u001f\n\u0007\n\n\u000e\n\u0001\n\u0005\n\u001a\n@\n\u001a\n\u0007\n\u001a\n(\n\n\u000e\n\u0001\n\u0007\n\u0016\n\u001a\n\b\n\u0016\n\u001a\n\u0007\n\n\u001d\n\u0007\n\u0016\n\b\n\u0016\n\b\n`\n\u0014\n\n(\n>\n6\n\u001b\n\n>\n\nA\n>\n6\n\u0007\nF\n\u0016\n(\n>\n(\n\u0010\nA\n\u0007\n\u0010\n\nA\n\u000f\n\u0006\nM\n1\n\u0013\n\u0005\nW\n\u0010\n\u000f\nW\n\u0011\n^\n\u0016\nb\n(\n\u001f\n\u0007\n\u001b\n>\n_\n\t\n(\n>\n\u001b\n\u001d\n\n\u0016\n@\n\u001a\nM\n\f\n1\n\u0013\n\u001b\n\u001b\n\u001b\n\b\nQ\n\u0015\n5\nQ\n\n\u001b\n`\n\n\u0015\n\u001b\n\u001b\n\u001b\nM\n\b\nQ\n\u0015\nM\n\f\n\u0016\n5\nQ\n\nM\n\u0016\n\u001f\n(\n\u001c\n\u0016\n\u0007\n>\n\u0010\n\u000f\nW\n\u0010\n\u000f\nW\n\u0011\n\u001d\n(\n\u0007\n\u0014\n\u0013\n\u001d\n@\n(\n\u0007\n\n\u001e\n\u0005\nW\n^\n\u0016\nb\n(\n\u001f\n\u0007\n\u001b\n\n(\n>\n6\n\u001b\n\n>\n\nA\n>\n6\n\u0007\nF\n\u0015\n\f(a)\n\nw(f )\n\n0\n\n(b)\n\nJ\n\n0.1\n\n\u22122\n\n0.05\n[deg\u22122]\n\n\u22122\n\n0\n\n2\n\n0\n0\n\n100\n\n300\n\n400\n\n200\nN\n\n\u0016?>\n\nshown by the dashed line.\n\nbut saturates in the limit of large\n. The precise form of\n\n. Figure 4 shows that for this range of\nthe deviations of the inverse square error from linearity are small. Thus, in\nis given by the asymptotic behavior, Eq. (19),\n\nby an approximate bilinear weight\n. The asymptotic result, Eq. (19), is smaller\nthan the optimal value given by the full FI, Eq. (11), see Fig. 4 (dotted line). In fact, it is\n\nof one over the squared estimation error, for the optimal bilinear readout in the multiplica-\ntive modulation model (open circles). The dashed line is the asymptotic behavior, given by\nEq. (19). Here\nmodulation model. The dotted line is the FI bound. In these simulations \u001c\n\n\u001b , Eq. (17), for the OBE with\n\u0002 . (b) Numerical evaluation\nFigure 3: (a) Pro\ufb01le of\u0010\nM , for the optimal bilinear readout in the multiplicative\n\u0001&\u001b\n\u0002 ,E\nandL\n\t\u0004\u0003\n were used.\ninverse square error initially increases linearly with\n. However, the saturation size\nincreases rapidly with\u001c\n\u001b depends on the speci\ufb01cs of the correlation model. For the exponentially decaying\ncorrelations assumed in Eq. (2), we \ufb01nd\n\u0013\u0012\u001c\u0002\u0001\n, and\u001c\nthe regime\u001f\n\u001b ,\n\n\u0001:\u001b\nWe thus conclude that the OBE (with unlimited\u001c ) will generate an inverse square estima-\ntion error which increases linearly with with a coef\ufb01cient given by Eq. (19), and that\nthis value can be achieved for reasonable values of\nmatrix, of the form of Eq. (20), with small\u001c\nequal to the error of an independent population with a variance which equals5\n\u0001\u001c\u001b and a\n\u0015\u0004\u0003\n\u0011\u0014\u0013\n(21) is very inef\ufb01cient, yielding a \ufb01nite error for large\nmatrix, Eq. (20), we rewrite Eq. (15) with\u0002\n\u000b\f\r?`\n\u000b\f\n\n76\n\nIt is important to note that in the presence of correlations, the quadratic readout of Eq.\nas shown in Fig. 4 (line marked\n\nTo understand the reason for the simple form of the approximately optimal bilinear weight\n\n(21)\n\n(22)\n\n(23)\n\n\u2018quadratic\u2019).\n\n5 Discussion\n\nquadratic population vector readout of the form\n\n\u0011\u0014\u0013\n60\u0011\u0014\u0013\n\n\u000b\f\n\nof Eq. (20) as\n\n\u000b26\n\n\u0015\u0004\u0003\n\n\u0015\u0006\u0005\n\nD\nq\nf\n\u001d\n\u0002\n\n\u0001\n\u001d\n\u0016\n\n\u0016\n@\n\u001a\nM\n\f\n\u001b\n\u0013\n\u0006\n\u001d\n\u0002\n\u0015\n\u001d\n\u001f\n\u001d\n\nO\nV\n-\n\u0016\n\u001c\n\u001b\n\nO\nV\n-\n\u0016\n\u001c\nO\nV\n-\n\n\u001d\n \n\u0019\n\u0019\n\n\u0019\n\u0019\n\nO\nV\n-\n\u0016\n\u001c\n\u0016\n@\n\u001a\nM\n\f\nQ\n\n\u0016\n\u001a\n\u0007\n\u0013\n\u000f\n\u0005\n\n\u000b\nM\n\n`\n\n\u001a\n\u0007\n\u001d\n\u000f\n\u0005\n\n\u001a\n\n\u0015\n\u0003\n\u001a\n\u001d\n\u000f\n\u0005\n\u001d\n@\n(\n\u001f\n\n\u001e\n\u0005\nW\n\u0011\n1\n\u001e\n`\n\nW\n\u000f\n1\n\u0011\n\u001f\n\f0.5\n\n0.4\n\n0.3\n\n0.2\n\n2\n\u2212\n\ng\ne\nd\n\n \n \n \n\n2\n\n d\n/\n\n1\n\n0.1\n\nJ\n\np=3\n\np=2\n\np=1\n\n0\n0\n\n500\n\n1000\nN\n\nquadratic\n\n1500\n\n2000\n\n\u0015\u0002\u0015\n\n\u001f andL\n\nComparing this form with Eq. (21) it can be seen that our readout is in the form of a bilinear\n\nline is the asymptotic behavior, given by Eq. (19). The FI bound is shown by the dotted\nline. For the simulations \u001c\n\nFigure 4: Inverse square estimation error of the \ufb01nite-\u001c approximation for the OBE, Eq.\n . The bottom curve is\u001c\u0006\u001d\b\u0002 . The dashed\n(20). Solid curves from the bottom\u001c\u0006\u001d\n\u0002 ,E\n\t\u0004\u0003\n were used.\npopulation vector in which the lowest Fourier modes of the response vector\u0018 have been\n\r because\ncross-correlations between the different components of the residual responses \u001a\n\u0001\u001c\u001b . In other words, the variance of a correlation pro\ufb01le which has only high Fourier\n\nthe underlying correlations have smooth spatial dependence, whose power is concentrated\nmostly in the low Fourier modes. On the other hand, the information contained in the\nvariance is not removed because the variance contains a discontinuous spatial component,\n\nmodes can still preserve its slowly varying components. Thus, by projecting out the low\nFourier modes of the spatial responses the spatial correlations are suppressed but the infor-\nmation in the response variance is retained.\n\nremoved. Retaining only the high Fourier modes in the response pro\ufb01le suppresses the\n\nThis interpretation of the bilinear readout implies that although all the elements of the\ncorrelation matrix depend on the stimulus, only the stimulus dependence of the diagonal\nelements is important. This important conclusion is borne out by our theoretical results\nconcerning the performance of the system. As Eqs. (11) and (19) show, the asymptotic\nperformance of both the full FI as well as that of the OBE are equivalent to those of an\n\nuncorrelated population with a stimulus dependent variance which equals5\nhave assumed here that the mean responses are untuned,*\n\nAlthough we have presented results here concerning a multiplicative model of correlations,\nwe have studied other models of stimulus dependent correlations. These studies indicate\nthat the above conclusions apply to a broad class of populations in which information is\nencoded in the second order statistics of the responses. Also, for the sake of clarity we\n. Our studies have shown\nthat adding tuned mean inputs does not modify the picture since the smoothly varying\npositive correlations greatly suppress the information embedded in the \ufb01rst order statistics.\n\n\u0001\u001c\u001b .\n\nThe relatively simple form of the readout Eq. (22) suggests that neuronal hardware may\nbe able to extract ef\ufb01ciently information embedded in local populations of cells whose\nnoisy responses are strongly correlated, provided that the variances of their responses are\nsigni\ufb01cantly tuned to the stimulus. This latter condition is not too restrictive, since tuning\nof variances of neuronal \ufb01ring rates to stimulus and motor variables is quite common in the\nnervous system.\n\nq\n\u001f\n\t\n\u0007\n\u0015\n\u001d\n\u0002\n\u0015\n\u001d\n\u001d\n\u000b\n5\nQ\n\n\u0016\nQ\n\n\u0016\n\u001d\n\u0011\n\fAcknowledgments\n\nThis work was partially supported by grants from the Israel-U.S.A. Binational Science\nFoundation and the Israeli Science Foundation. M.S. is supported by a scholarship from\nthe Clore Foundation.\n\nReferences\n\n[1] E. Fetz, K. Yoyoma and W. Smith, Cerebral Cortex\n\n1991).\n\n(Plenum Press, New York,\n\n[2] D. Lee, N.L. Port, W. Kruse and A.P. Georgopoulos, J. Neurosci.\n[3] E.M. Maynard, N.G. Hatsopoulos, C.L. Ojakangas, B.D. Acuna, J.N. Sanes, R.A.\n\n, 1161 (1998).\n\n\u0001\u0003\u0002\n\nNormann, and J.P. Donoghue, J. Neurosci. 19, 8083 (1999).\n\n[4] E. Zohary, M.N. Shadlen and W.T. Newsome, Nature\n[5] L.F. Abbott and P. Dayan, Neural Computation\n[6] H. Sompolinsky, H. Yoon, K. Kang and M. Shamir, Phys. Rev. E,\n\n, 91 (1999).\n\n\u0004\u0006\u0005\n\n\u0001\u0007\u0001\n\n, 140 (1994).\n\n, 051904 (2001);\nH. Yoon and H. Sompolinsky, Advances in Neural Information Processing Systems\n11 (pp. 167). Kearns M.J, Solla S.A and Cohn D.A, Eds., (Cambridge, MA: MIT\nPress, 1999).\n\n\b\n\t\n\n[7] S. Seung and H. Sompolinsky, Proc. Natl. Acad. Sci. USA\n[8] E. Salinas and L.F. Abbott, J. Comp. Neurosci.\n\n, 89 (1994).\n\n, 10794 (1993).\n\n\n\u0011\n\n\u0011\n\u0001\n\f", "award": [], "sourceid": 2031, "authors": [{"given_name": "Maoz", "family_name": "Shamir", "institution": null}, {"given_name": "Haim", "family_name": "Sompolinsky", "institution": null}]}