{"title": "Characterizing neural dependencies with copula models", "book": "Advances in Neural Information Processing Systems", "page_first": 129, "page_last": 136, "abstract": "The coding of information by neural populations depends critically on the statistical dependencies between neuronal responses. However, there is no simple model that combines the observations that (1) marginal distributions over single-neuron spike counts are often approximately Poisson; and (2) joint distributions over the responses of multiple neurons are often strongly dependent. Here, we show that both marginal and joint properties of neural responses can be captured using Poisson copula models. Copulas are joint distributions that allow random variables with arbitrary marginals to be combined while incorporating arbitrary dependencies between them. Different copulas capture different kinds of dependencies, allowing for a richer and more detailed description of dependencies than traditional summary statistics, such as correlation coefficients. We explore a variety of Poisson copula models for joint neural response distributions, and derive an efficient maximum likelihood procedure for estimating them. We apply these models to neuronal data collected in and macaque motor cortex, and quantify the improvement in coding accuracy afforded by incorporating the dependency structure between pairs of neurons.", "full_text": "CharacterizingneuraldependencieswithcopulamodelsPietroBerkesVolenCenterforComplexSystemsBrandeisUniversity,Waltham,MA02454berkes@brandeis.eduFrankWoodandJonathanPillowGatsbyComputationalNeuroscienceUnit,UCLLondonWC1N3AR,UK{fwood,pillow}@gatsby.ucl.ac.ukAbstractThecodingofinformationbyneuralpopulationsdependscriticallyonthestatisti-caldependenciesbetweenneuronalresponses.However,thereisnosimplemodelthatcansimultaneouslyaccountfor(1)marginaldistributionsoversingle-neuronspikecountsthatarediscreteandnon-negative;and(2)jointdistributionsovertheresponsesofmultipleneuronsthatareoftenstronglydependent.Here,weshowthatbothmarginalandjointpropertiesofneuralresponsescanbecapturedusingcopulamodels.Copulasarejointdistributionsthatallowrandomvariableswitharbitrarymarginalstobecombinedwhileincorporatingarbitrarydependenciesbe-tweenthem.Differentcopulascapturedifferentkindsofdependencies,allowingforaricherandmoredetaileddescriptionofdependenciesthantraditionalsum-marystatistics,suchascorrelationcoef\ufb01cients.Weexploreavarietyofcopulamodelsforjointneuralresponsedistributions,andderiveanef\ufb01cientmaximumlikelihoodprocedureforestimatingthem.Weapplythesemodelstoneuronaldatacollectedinmacaquepre-motorcortex,andquantifytheimprovementincod-ingaccuracyaffordedbyincorporatingthedependencystructurebetweenpairsofneurons.We\ufb01ndthatmorethanonethirdofneuronpairsshowsdependencyconcentratedintheloweroruppertailsfortheir\ufb01ringratedistribution.1IntroductionAnimportantprobleminsystemsneuroscienceistodevelop\ufb02exible,statisticallyaccuratemodelsofneuralresponses.ThestochasticspikingactivityofindividualneuronsincortexisoftenwelldescribedbyaPoissondistribution.Responsesfrommultipleneuronsalsoexhibitstrongdependen-cies(i.e.,correlations)duetosharedinputnoiseandlateralnetworkinteractions.However,thereisnonaturalmultivariategeneralizationofthePoissondistribution.Forthisreason,muchofthelitera-tureonpopulationcodinghastendedeithertoignorecorrelationsentirely,treatingneuralresponsesasindependentPoissonrandomvariables[1,2],ortoadoptaGaussianmodelofjointresponses[3,4],assumingaparametricformfordependenciesbutignoringkeyfeatures(e.g.,discreteness,non-negativity)ofthemarginaldistribution.Recentworkhasfocusedontheconstructionoflargeparametricmodelsthatcaptureinter-neuronaldependenciesusinggeneralizedlinearpoint-processmodels[5,6,7,8,9]andbinarysecond-ordermaximum-entropymodels[10,11,12].Althoughtheseapproachesarequitepowerful,theymodelspiketrainsonlyinvery\ufb01netimebins,andthusdescribethedependenciesinneuralspikecountdistributionsonlyimplicitly.Modelingthejointdistributionofneuralactivitiesisthereforeanimportantopenproblem.Hereweshowhowtoconstructnon-independentjointdistributionsover\ufb01ringratesusingcopulas.Inparticular,thisapproachcanbeusedtocombinearbitrarymarginal\ufb01ringratedistributions.Thedevelopmentofthepaperisasfollows:inSection2,weprovideabasicintroductiontocopulas;inSection3,wederiveamaximumlikelihoodestimationprocedureforneuralcopulamodels,inSections4and5,weapplythesemodelstophysiologicaldatacollectedinmacaquepre-motor1\f\uf021\uf022Figure1:Samplesdrawnfromajointdistributionde\ufb01nedusingthedependencystructureofabivariateGaus-siandistributionandchangingthemarginaldistributions.Toprow:Themarginaldistributions(theleftmostmarginalisuniform,byde\ufb01nitionofcopula).Bottomrow:Thelog-densityfunctionofaGaussiancopula,andsamplesfromthejointdistributionde\ufb01nedasinEq.2.cortex;\ufb01nally,inSection6wereviewtheinsightsprovidedbyneuralcopulamodelsanddiscussseveralextensionsandfuturedirections.2CopulasAcopulaC(u1,...,un):[0,1]n\u2192[0,1]isamultivariatedistributionfunctionontheunitcubewithuniformmarginals[13,14].Thebasicideabehindcopulasisquitesimple,andiscloselyrelatedtothatofhistogramequalization:forarandomvariableyiwithcontinuouscumulativedistributionfunction(cdf)Fi,therandomvariableui:=Fi(yi)isuniformlydistributedontheinterval[0,1].Onecanusethisbasicpropertytoseparatethemarginalsfromthedependencystructureinamul-tivariatedistribution:thefullmultivariatedistributionisstandardizedbyprojectingeachmarginalontooneaxisoftheunithyper-cube,andleavingonewithadistributiononthehyper-cube(thecop-ula,byde\ufb01nition)thatrepresentdependenciesinthemarginals\u2019quantiles.ThisintuitionhasbeenformalizedinSklar\u2019sTheorem[15]:Theorem1(Sklar,1959)Givenu1,...,unrandomvariableswithcontinuousdistributionfunc-tionsF1,...,FnandjointdistributionF,thereexistauniquecopulaCsuchthatforallui:C(u1,...,un)=F(F\u221211(u1),...,F\u22121n(un))(1)Conversely,givenanydistributionfunctionsF1,...,FnandcopulaC,F(y1,...,yn)=C(F1(y1),...,Fn(yn))(2)isan-variatedistributionfunctionwithmarginaldistributionfunctionsF1,...,Fn.Thisresultgivesawaytoderiveacopulagiventhejointandmarginaldistributions(usingEq.1),andalso,moreimportantlyhere,toconstructajointdistributionbyspecifyingthemarginaldistributionsandthedependencystructureseparately(Eq.2).Forexample,onecankeepthedependencystructure\ufb01xedandvarythemarginals(Fig.1),orviceversagiven\ufb01xedmarginaldistributionsde\ufb01nenewjointdistributionsusingparametrizedcopulafamilies(Fig.2).Forillustration,inthispaperwearegoingtoconsideronlythebivariatecase.Allthemethods,however,generalizestraightforwardlytothemultivariatecase.Sincecopulasdonotdependonthemarginals,onecande\ufb01neinthiswaydependencymeasuresthatareinsensitivetonon-lineartransformationsoftheindividualvariables[14]andgeneralizecorrela-tioncoef\ufb01cients,whichareonlyappropriateforellipticdistributions.Thecopularepresentationhasalsobeenusedtoestimatetheconditionalentropyofneurallatenciesbyseparatingthecontributionoftheindividuallatenciesfromthatcomingfromtheircorrelations[16].Dependenciesstructuresarespeci\ufb01edbyparametriccopulafamilies.OnenotableexampleistheGaussiancopula,whichgeneralizesthedependencystructureofthemultivariateGaussiandistribu-tiontoarbitrarymarginaldistribution(Fig.1),andisde\ufb01nedasC(u1,u2;\u03a3)=\u03a6\u03a3!\u03c6\u22121(u1),\u03c6\u22121(u2)\",(3)2\fFigure2:Samplesdrawnfromajointdistributionwith\ufb01xedGaussianmarginalsanddependencystructurede\ufb01nedbyparametriccopulafamilies,asindicatedbythelabels.Toprow:log-densityfunctionforthreecopulafamilies.Bottomrow:Samplesfromthejointdistribution(Eq.2).GaussianCN\u03a3(u1,u2)=\u03a6\u03a3!\u03c6\u22121(u1),\u03c6\u22121(u2)\"FrankCFr\u03b8(u1,u2)=\u22121\u03b8log#1+(e\u2212\u03b8u1\u22121)(e\u2212\u03b8u2\u22121)e\u2212\u03b8\u22121$ClaytonCCl\u03b8(u1,u2)=(u\u2212\u03b81+u\u2212\u03b82\u22121)\u22121/\u03b8,\u03b8>0ClaytonnegativeCNeg\u03b8(u1,u2)=max%(u\u2212\u03b81+u\u2212\u03b82\u22121),0&\u22121/\u03b8,\u22121\u2264\u03b8<0GumbelCGu\u03b8(u1,u2)=exp!\u2212(\u02dcu\u03b81+\u02dcu\u03b82)1/\u03b8\",\u02dcuj=\u2212loguj,\u03b8\u22651Table1:De\ufb01nitionoffamiliesofcopuladistributionfunctions.where\u03c6(u)isthecdfoftheunivariateGaussianwithmean0andvariance1,and\u03a6\u03a3isthecdfofastandardmultivariateGaussianwithmean0andcovariancematrix\u03a3.Otherfamiliesderivefromtheeconomicsliterature,andaretypicallyone-parameterfamiliesthatcapturevariouspossibledependencies,forexampledependenciesonlyinoneofthetailsofthedistribution.Table1showsthede\ufb01nitionofthecopuladistributionsusedinthispaper(see[14],foranoverviewofknowncopulasandcopulaconstructionmethods).3MaximumLikelihoodestimationfordiscretemarginaldistributionsInthecasewheretherandomvariableshavediscretedistributionfunctions,asinthecaseofneural\ufb01ringrates,onlyaweakerversionofTheorem1isvalid:therealwaysexistsacopulathatsatis\ufb01esEq.2,butitisnolongerguaranteedtobeunique[17].Withdiscretedata,theprobabilityofaparticularoutcomeisdeterminedbyanintegralovertheregionof[0,1]ncorrespondingtothatoutcome;anytwocopulasthatintegratetothesamevaluesonallsuchregionsproducethesamejointdistribution.WecanderiveaMaximumLikelihood(ML)estimationoftheparameters\u03b8byconsideringagener-ativemodelwhereuniformmarginalsaregeneratedfromthecopuladensity,andinturnusethesetogeneratethediscretevariablesdeterministicallyusingtheinverse(marginal)distributionfunctions,asinFig.3.Thesemarginalscanbegivenbytheempiricalcumulativedistributionof\ufb01ringrates(asinthispaper)orbyanyparametrizedfamilyofunivariatedistributions(suchasPoisson).TheMLequationthenbecomesargmax\u03b8p(y|\u03b8)=argmax\u03b8\u2019p(y|u)p(u|\u03b8)du(4)=argmax\u03b8\u2019F1(y1)F1(y1\u22121)\u00b7\u00b7\u00b7\u2019Fn(yn)Fn(yn\u22121)c\u03b8(u1,...,un)du,(5)3\f\u03b8uy\u03bbp(u|\u03b8)=c\u03b8(u1,...,un)p(yi|u,\u03bb)=!1,yi=F\u22121i(ui;\u03bbi)0,otherwiseFigure3:Graphicalrepresentationofthecopulamodelwithdiscretemarginals.Uniformmarginalsuaredrawnfromthecopuladensityfunctionc\u03b8(u1,...,un),parametrizedby\u03b8.Thediscretemarginalsarethengenerateddeterministicallyusingtheinversecdfofthemarginals,whichareparametrizedby\u03bb.\u22121\u22120.500.51\u22120.300.3Gaussian012345\u22120.300.3Clayton12345678910\u22120.300.3Gumbel\u221210\u221250510\u22120.300.3FrankFigure4:Distributionofthemaximumlikelihoodestimationoftheparametersoffourcopulafamilies,forvarioussettingoftheirparameter(x-axis).Onthey-axis,estimatesarecenteredsuchthat0correspondstoanunbiasedestimate.Errorbarsareonestandarddeviationoftheestimate.whereFicandependonadditionalparameters\u03bbi.Thelastequationisthecopulaprobabilitymassinsidethevolumede\ufb01nedbytheverticesFi(yi)andFi(yi\u22121),andcanbereadilycomputedusingthecopuladistributionC\u03b8(u1,...,un).Forexample,inthebivariatecaseoneobtainsargmax\u03b8p(y1,y2|\u03b8)=argmax\u03b8(C\u03b8(u1,u2)+C\u03b8(u\u22121,u\u22122)\u2212C\u03b8(u\u22121,u2)\u2212C\u03b8(u1,u\u22122)),(6)whereui=Fi(yi)andu\u2212i=Fi(yi\u22121).MLoptimizationcanbeperformedusingstandardmethods,likegradientdescent.Inthebivariatecase,we\ufb01ndthatoptimizationusingthestandardMATLABoptimizationroutinesisrelativelyef-\ufb01cient.Givenneuraldataintheformof\ufb01ringratesy1,y2fromapairofneurons,wecollecttheempiricalcumulativehistogramofresponses,Fi(k)=P(yi\u2264k).Thedataisthentransformedthroughthecdfsui=Fi(yi),andthecopulamodelis\ufb01taccordingtoEq.6.Ifaparametricdistribu-tionfamilyisusedforthemarginals,theparametersofthecopula\u03b8andthoseofthemarginals\u03bbcanbeestimatedsimultaneously,oralternatively\u03bbcanbe\ufb01tted\ufb01rst,followedby\u03b8.Inourexperience,thesecondmethodismuchfasterandthequalityofthe\ufb01tistypicallyunchanged.WecheckedforbiasesinMLestimationduetoalimitedamountofdataandlow\ufb01ringratebygeneratingdatafromthediscretecopulamodel(Fig.3),foranumberofcopulafamiliesandPoissonmarginalswithparameters\u03bb1=2,\u03bb2=3.Theestimateisbasedon3500observationsgeneratedfromthemodels(1000fortheGaussiancopula).Theestimationisrepeated200times(100fortheGaussiancopula)inordertocomputethemeanandstandarddeviationoftheMLestimate.Figure4showsthattheestimateisunbiasedandaccurateforawiderangeofparameters.Inaccuracyintheestimationbecomeslargerasthecopulasapproachfunctionaldependency(i.e.,u2=f(u1)foradeterministicfunctionf),asitisthecasefortheGaussiancopulawhen\u03c1tendsto1,andfortheGumbelcopulaas\u03b8goestoin\ufb01nity.Thisisaneffectduetolow\ufb01ringrates.Givenourformulationoftheestimationproblemasagenerativemodel,onecouldusemoresophisticatedBayesianmethodsinplaceoftheMLestimation,inordertoinferawholedistributionoverparametersgiventhedata.Thiswouldallowtoputerrorbarsontheestimatedparameters,andwouldavoidover\ufb01ttingatthecostofcomputationaltime.4\f\uf021\uf022\uf023\uf022\uf024\uf022\uf025\uf022\uf026\uf022\uf027\uf022\uf028\uf029\uf02a\uf02b\uf02c\uf029\uf02d\uf02e\uf02f\uf02b\uf030\uf031\uf02d\uf02e\uf02a\uf02b\uf031\uf021\uf02f\uf026\uf02b\uf032\uf02d\uf033\uf02f\uf029\uf02a\uf031\uf021\uf034\uf028\uf029\uf02a\uf02b\uf02c\uf029\uf02d\uf02e\uf02f\uf02b\uf030\uf031\uf02d\uf02e\uf02a\uf02b\uf031\uf021\uf02f\uf026\uf02b\uf032\uf02d\uf033\uf02f\uf029\uf02a\uf031\uf021\uf034\uf035\uf034\uf036\uf02d\uf031\uf02d\uf024\uf021\uf037\uf02b\uf038\uf029\uf036\uf039\uf037\uf021\uf035\uf034\uf036\uf02d\uf031\uf02d\uf024\uf021\uf037\uf02b\uf038\uf029\uf036\uf039\uf037\uf021\uf03a\uf03b\uf03a\uf03c\uf039\uf03b\uf039\uf03c\uf03a\uf03b\uf03a\uf03c\uf039\uf03b\uf039\uf03cFigure5:Empiricaljointdistributionandcopula\ufb01tfortwoneuronpairs.Thetoprowshowstwoneuronsthathavedependenciesmainlyintheuppertailsoftheirmarginaldistribution.Thepairinthebottomrowhasnegativedependency.a,d)Histogramofthe\ufb01ringrateofthetwoneurons.Colorscorrespondtothelogarithmofthenormalizedfrequency.b,e)Empiricalcopula.Thecolorintensityhasbeencutoffat2.0toimprovevisibility.c,f)Densityofthecopula\ufb01t.4ResultsTodemonstratetheabilityofcopulamodelsto\ufb01tjoint\ufb01ringratedistribution,wemodelneuraldatarecordedusingamulti-electrodearrayimplantedinthepre-motorcortex(PMd)areaofamacaquemonkey[18,19].Thearrayconsistedin10\u00d710electrodesseparatedby400\u00b5m.Firingtimeswererecordedwhilethemonkeyexecutedacenter-outreachingtask.See[19]foradescriptionofthetaskandgeneralexperimentalsetup.We\ufb01tthecopulamodelusingthemarginaldistributionofneuralactivityovertheentirerecordingsession,includingdatarecordedbetweentrials(i.e.,whilethemonkeywasfreelybehaving).Althoughonemightalsoliketoconsiderdatacollectedduringasingletaskcondition(i.e.,thestimulus-conditionalresponsedistribution),themarginalresponsedistributionisanimportantstatisticalobjectinitsownright,andhasbeenthefocusofrecentmuchliterature[10,11].Forexample,thejointactivityacrossneurons,averagedoverstimuli,istheonlydistributionthebrainhasaccessto,andmustbesuf\ufb01cientforlearningtoconstructrepresentationsoftheexternalworld.Wecollectedspikeresponsesin100msbins,andselectedatrandom,withoutrepetition,atrainingsetof4000binsandatestsetof2000bins.Outofatotalof194neuronsweselectasubsetof33neuronsthat\ufb01redaminimumof2500spikesoverthewholedataset.Foreverypairofneuronsinthissubset(528pairs),we\ufb01ttheparametersofseveralcopulafamiliestothejoint\ufb01ringrate.Figure5showstwoexamplesofthekindofthedependenciespresentinthedatasetandhowtheyare\ufb01tbydifferentcopulafamilies.Theneuronpairinthetoprowshowsdependencyintheuppertailsoftheirdistribution,ascanbeseeninthehistogramofjoint\ufb01ringrates(colorsrepresentthelogarithmofthefrequency):Thetwoneuronshavethetendencyto\ufb01restronglytogether,butarerel-ativelyindependentatlow\ufb01ringrates.Thisiscon\ufb01rmedbytheempiricalcopula,whichshowstheprobabilitymassintheregionsde\ufb01nedbythecdfsofthemarginaldistribution.Sincethemarginalcdfsarediscrete,thedataisprojectedonadiscretesetofpointsontheunitcube;thecolorsintheempiricalcopulaplotsrepresenttheprobabilitymassintheregionwherethemarginalcdfsareconstant.Theaxisintheempiricalcopulashouldbeinterpretedasthequantilesofthemarginaldistributions\u2013forexample,0.5onthex-axiscorrespondstothemedianofthedistributionofy1.Thehigherprobabilitymassintheupperrightcorneroftheplotthusmeansthatthetwoneuronstendtobeintheuppertailsofthedistributionssimultaneously,andthustohavehigher\ufb01ringratestogether.Ontheright,onecanseethatthisdependencystructureiswellcapturedbytheGumbelcopula\ufb01t.Thesecondpairofneuroninthebottomrowhavenegativedependency,inthesensethatwhenoneofthemhashigh\ufb01ringratetheothertendstobesilent.Althoughthisisnotreadilyvisibleinthejointhistogram,thedependencybecomesclearintheempiricalcopulaplot.ThisstructureiscapturedbytheFrankcopula\ufb01t.5\f\u22120.500.51\u22124\u221220246Gauss parameterFrank parameter< 0010< 0010Gauss gain (bits/sec)Frank gain (bits/sec)Figure6:Inthepairswheretheir\ufb01timprovesovertheindependencemodel,theparameters(left)andthescore(right)oftheGaussianandFrankmodelsarehighlycorrelated.Thegoodness-of-\ufb01tofthecopulafamiliesisevaluatedbycross-validation:We\ufb01tdifferentmodelsontrainingdata,andcomputethelog-likelihoodoftestdataunderthe\ufb01ttedmodel.Themodelsarescoredaccordingtothedifferencebetweenthelog-likelihoodofamodelthatassumesindependentneuronsandthelog-likelihoodofthecopulamodel.Thismeasure(appropriatelyrenormalized)canbeinterpretedasthenumberofbitspersecondthatcanbesavedwhencodingthe\ufb01ringratebytakingintoaccountthedependenciesencodedbythecopulafamily.ThisisbecausethisquantitycanbeexpressedasanestimationofthedifferenceintheKullback-Leiblerdivergenceoftheindependent(pindep)andcopulamodel(p\u03b8)totherealdistributionp\u2217&logp\u03b8(y)\u2019y\u223cp\u2217\u2212&logpindep(y)\u2019y\u223cp\u2217(7)\u2248\u2019p\u2217(y)logp\u03b8(y)dy\u2212\u2019p\u2217(y)logpindep(y)(8)=KL(p\u2217||pindep)\u2212KL(p\u2217||p\u03b8).(9)Wetookparticularcareinselectingasmallsetofcopulafamiliesthatwouldbeabletocapturethedependenciesoccurringinthedata.Someofthefamiliesthatweconsideredat\ufb01rstcapturesimilarkindofdependencies,andtheirscoresarehighlycorrelated.Forexample,theFrankandGaussiancopulasareabletorepresentbothpositiveandnegativedependenciesinthedata,andsimultaneouslyinloweranduppertails,althoughthedependenciesinthetailsarelessstrongfortheFrankfamily(comparethecopuladensitiesinFigs.1and5f).Fig.6(left)showsthatboththeparameter\ufb01tsandtheirperformancearehighlycorrelated.AnadvantageoftheFrankcopulaisthatitismuchmoreef\ufb01cientto\ufb01t,sincetheGaussiancopularequiresmultipleevaluationsofthebivariateGaussiancdf,whichrequiresexpensivenumericalcalculations.Inaddition,TheGaussiancopulawasalsofoundtobemorepronetoover\ufb01ttingonthisdataset(Fig.6,right).Forthesereasons,wedecidedtousetheFrankfamilyonlyfortherestoftheanalysis.Withsimilarproceduresweshortlistedatotal3familiesthatcoverthevastmajorityofdependenciesinourdataset:Frank,Clayton,andGumbelcopulas.ExamplesofthecopuladensityofthesefamiliescanbefoundinFigs.2,and5.TheClaytonandGumbelcopulasdescribedependenciesintheloweranduppertailsofthedistributions,respectively.Wedidn\u2019t\ufb01ndanyexampleofneuronpairswherethedependencywouldbeintheuppertailofthedistributionforoneandinthelowertailfortheotherdistribution,ormorecomplicateddependencies.Outofall528neuronpairs,393hadasigni\ufb01cantimprovement(P<0.05ontestdata)overamodelwithindependentneurons1andfor102pairstheimprovementwaslargerthan1bit/sec.Dependen-ciesinthedatasetseemthustobewidespread,despitethefactthatindividualneuronsarerecordedfromelectrodesthatareupto4.4mmapart.Fig.7showsthehistogramofimprovementinbits/sec.ThemostcommondependenciesstructuresoverallneuronpairsaregivenbytheGaussian-likede-pendenciesoftheFrankcopula(54%ofthepairs).Interestingly,alargeproportionoftheneuronsshoweddependenciesconcentratedintheuppertails(Gumbelcopula,22%)orlowertails(Claytoncopula,16%)ofthedistributions(Fig.7).1Wecomputedthesigni\ufb01cancelevelbygeneratinganarti\ufb01cialdatasetusingindependentneuronswiththesameempiricalpdfasthemonkeydata.Weanalyzedthegenerateddataandcomputedthemaximalimprove-mentoveranindependentmodel(duetothelimitednumberofsamples)onarti\ufb01cialtestdata.Theresultingdistributionisverynarrowlydistributedaroundzero.Wetookthe95thpercentileofthedistribution(0.02bits/sec)asthethresholdforsigni\ufb01cance.6\f051015020406080100bits/secnumber of pairsImprovement over independent model<\u2212 371Independent 8%Clayton  15%Gumbel  23%Frank  54%Figure7:Foreverypairofneurons,weselectthecopulafamilythatshowsthelargestimprovementoveramodelwithindependentneurons,inbits/sec.Left:histogramofthegaininbits/secovertheindependentmodel.Right:Piechartofthecopulafamiliesthatbest\ufb01ttheneuronpairs.5DiscussionTheresultspresentedhereshowthatitispossibletorepresentneuronalspikeresponsesusingamodelthatpreservesdiscrete,non-negativemarginalswhileincorporatingvarioustypesofdepen-denciesbetweenneurons.Mathematically,itisstraightforwardtogeneralizethesemethodstothen-variatecase(i.e.,distributionsovertheresponsesofnneurons).However,manycopulafamilieshaveonlyoneortwoparameters,regardlessofthecopuladimensionality.Ifthedependencystruc-tureacrossaneuralpopulationisrelativelyhomogeneous,thenthesecopulasmaybeusefulinthattheycanbeestimatedusingfarlessdatathanrequired,e.g.,forafullcovariancematrix(whichhasO(n2)parameters).Ontheotherhand,ifthedependencieswithinapopulationvarymarkedlyfordifferentpairsofneurons(asinthedatasetexaminedhere),suchcopulaswilllackthe\ufb02exibilitytocapturethecomplicateddependencieswithinafullpopulation.Insuchcases,wecanstillapplytheGaussiancopula(andothercopulasderivedfromellipticallysymmetricdistributions),sinceitisparametrizedbythesamecovariancematrixasan-dimensionalGaussian.However,theGaussiancopulabecomesprohibitivelyexpensiveto\ufb01tinhighdimensions,sinceevaluatingthelikelihoodrequiresanexponentialnumberofevaluationsofthemultivariateGaussiancdf,whichitselfmustbecomputednumerically.Onechallengeforfutureworkwillthereforebetodesignnewparametricfamiliesofcopulaswhoseparametersgrowwiththenumberofneurons,butremaintractableenoughformaximum-likelihoodestimation.Recently,Kirshner[20]proposedacopula-basedrepresentationformultivariatedistri-butionsusingamodelthataveragesovertree-structuredcopuladistributions.Thebasicideaisthatpairwisecopulascanbeeasilycombinedtoproduceatree-structuredrepresentationofamultivari-atedistribution,andthataveragingoversuchtreesgivesanevenmore\ufb02exibleclassofmultivariatedistributions.Weplantoexaminethisapproachusingneuralpopulationdatainfuturework.Anotherfuturechallengeistocombineexplicitmodelsofthestimulus-dependenceunderlyingneu-ralresponseswithmodelscapableofcapturingtheirjointresponsedependencies.Thedatasetanalyzedhereconcernedthedistributionoverspikeresponsesduringallallstimulusconditions(i.e.,themarginaldistributionoverresponses,asopposedtothetheconditionalresponsedistribu-tiongivenastimulus).Althoughthismarginalresponsedistributionisinterestinginitsownright,formanyapplicationsoneisinterestedinseparatingcorrelationsthatareinducedbyexternalstimulifrominternalcorrelationsduetothenetworkinteractions.OneobviousapproachistoconsiderahybridmodelwithaLinear-Nonlinear-Poissonmodel[21]capturingstimulus-inducedcorrelation,adjoinedtoacopuladistributionthatmodelstheresidualdependenciesbetweenneurons(Fig.8).Thisisanimportantavenueforfutureexploration.AcknowledgmentsWe\u2019dliketothankMatthewFellowsforprovidingthedatausedinthisstudy.Thisworkwassup-portedbytheGatsbyCharitableFoundation.7\fFigure8:HybridLNP-copulamodel.TheLNPpartofthemodelremovesstimulus-inducedcorrelationsfromtheneuraldata,sothatthecopulamodelcantakeintoaccountresidualnetwork-relateddependencies.References[1]R.Zemel,P.Dayan,andA.Pouget.Probabilisticinterpretationofpopulationcodes.NeuralComputation,10:403\u2013430,1998.[2]A.Pouget,K.Zhang,S.Deneve,andP.E.Latham.Statisticallyef\ufb01cientestimationusingpopulationcoding.NeuralComputation,10(2):373\u2013401,1998.[3]L.AbbottandP.Dayan.Theeffectofcorrelatedvariabilityontheaccuracyofapopulationcode.NeuralComputation,11:91\u2013101,1999.[4]E.Maynard,N.Hatsopoulos,C.Ojakangas,B.Acuna,J.Sanes,R.Normann,andJ.Donoghue.Neu-ronalinteractionsimprovecorticalpopulationcodingofmovementdirection.JournalofNeuroscience,19:8083\u20138093,1999.[5]E.Chornoboy,L.Schramm,andA.Karr.Maximumlikelihoodidenti\ufb01cationofneuralpointprocesssystems.BiologicalCybernetics,59:265\u2013275,1988.[6]W.Truccolo,U.T.Eden,M.R.Fellows,J.P.Donoghue,andE.N.Brown.Apointprocessframeworkforrelatingneuralspikingactivitytospikinghistory,neuralensembleandextrinsiccovariateeffects.J.Neurophysiol,93(2):1074\u20131089,2004.[7]M.Okatan,M.Wilson,andE.Brown.Analyzingfunctionalconnectivityusinganetworklikelihoodmodelofensembleneuralspikingactivity.NeuralComputation,17:1927\u20131961,2005.[8]S.Gerwinn,J.H.Macke,M.Seeger,andM.Bethge.Bayesianinferenceforspikingneuronmodelswithasparsityprior.AdvancesinNeuralInformationProcessingSystems,2008.[9]J.W.Pillow,J.Shlens,L.Paninski,A.Sher,A.M.Litke,andE.P.Chichilnisky,E.J.Simoncelli.Spatio-temporalcorrelationsandvisualsignalinginacompleteneuronalpopulation.Nature,454(7206):995\u2013999,2008.[10]E.Schneidman,M.Berry,R.Segev,andW.Bialek.Weakpairwisecorrelationsimplystronglycorrelatednetworkstatesinaneuralpopulation.Nature,440:1007\u20131012,2006.[11]J.Shlens,G.Field,J.Gauthier,M.Grivich,D.Petrusca,A.Sher,LitkeA.M.,andE.J.Chichilnisky.Thestructureofmulti-neuron\ufb01ringpatternsinprimateretina.JNeurosci,26:8254\u20138266,2006.[12]J.H.Macke,P.Berens,A.S.Ecker,A.S.Tolias,andM.Bethge.Generatingspiketrainswithspeci\ufb01edcorrelationcoef\ufb01cients.NeuralComputation,21(2),2009.[13]H.Joe.Multivariatemodelsanddependenceconcepts.Chapman&Hall,London,1997.[14]R.B.Nelsen.Anintroductiontocopulas.Springer,NewYork,2ndedition,2006.[15]A.Sklar.Fonctionsder\u00e9partition\u00e0ndimensionsetleursmarges.Publ.Inst.Statist.Univ.Paris,8:229\u2013231,1959.[16]R.L.JenisonandR.A.Reale.Theshapeofneuraldependence.NeuralComputation,16(4):665\u2013672,2004.[17]C.GenestandJ.Neslehova.Aprimeroncopulasforcountdata.AstinBulletin,37(2):475\u2013515,2007.[18]M.Serruya,N.Hatsopoulos,L.Paninski,M.Fellows,andJ.Donoghue.Instantneuralcontrolofamovementsignal.Nature,416:141\u2013142,2002.[19]S.Suner,MRFellows,C.Vargas-Irwin,GKNakata,andJPDonoghue.Reliabilityofsignalsfromachronicallyimplanted,silicon-basedelectrodearrayinnon-humanprimateprimarymotorcortex.NeuralSystemsandRehabilitationEngineering,IEEETransactionson,13(4):524\u2013541,2005.[20]S.Kirshner.Learningwithtree-averageddensitiesanddistributions.NIPS,20,2008.[21]E.P.Simoncelli,L.Paninski,J.Pillow,andO.Schwartz.Characterizationofneuralresponseswithstochasticstimuli.InM.Gazzaniga,editor,TheCognitiveNeurosciences,pages327\u2013338.MITPress,3rdedition,2004.8\f", "award": [], "sourceid": 856, "authors": [{"given_name": "Pietro", "family_name": "Berkes", "institution": null}, {"given_name": "Frank", "family_name": "Wood", "institution": null}, {"given_name": "Jonathan", "family_name": "Pillow", "institution": null}]}