{"title": "Exploring Functional Connectivities of the Human Brain using Multivariate Information Analysis", "book": "Advances in Neural Information Processing Systems", "page_first": 270, "page_last": 278, "abstract": "In this study, we present a method for estimating the mutual information for a localized pattern of fMRI data. We show that taking a multivariate information approach to voxel selection leads to a decoding accuracy that surpasses an univariate inforamtion approach and other standard voxel selection methods. Furthermore,we extend the multivariate mutual information theory to measure the functional connectivity between distributed brain regions. By jointly estimating the information shared by two sets of voxels we can reliably map out the connectivities in the human brain during experiment conditions. We validated our approach on a 6-way scene categorization fMRI experiment. The multivariate information analysis is able to \ufb01nd strong information \ufb02ow between PPA and RSC, which con\ufb01rms existing neuroscience studies on scenes. Furthermore, by exploring over the whole brain, our method identifies other interesting ROIs that share information with the PPA, RSC scene network,suggesting interesting future work for neuroscientists.", "full_text": "Exploring Functional Connectivity of the Human\nBrain using Multivariate Information Analysis\n\nBarry Chai1\u2217\n\nDirk B. Walther2\u2217\n\nDiane M. Beck2,3\u2020\n\nLi Fei-Fei1\u2020\n\n1Computer Science Department, Stanford University, Stanford, CA 94305\n\n2Beckman Institute, University of Illinois at Urbana-Champaign, Urbana, IL 61801\n\n3Psychology Department, University of Illinois at Urbana-Champaign, Champaign, IL 61820\n{bwchai,feifeili}@cs.stanford.edu {walther,dmbeck}@illinois.edu\n\nAbstract\n\nIn this study, we present a new method for establishing fMRI pattern-based\nfunctional connectivity between brain regions by estimating their multivariate\nmutual information. Recent advances in the numerical approximation of high-\ndimensional probability distributions allow us to successfully estimate mutual\ninformation from scarce fMRI data. We also show that selecting voxels based\non the multivariate mutual information of local activity patterns with respect to\nground truth labels leads to higher decoding accuracy than established voxel selec-\ntion methods. We validate our approach with a 6-way scene categorization fMRI\nexperiment. Multivariate information analysis is able to \ufb01nd strong information\nsharing between PPA and RSC, consistent with existing neuroscience studies on\nscenes. Furthermore, an exploratory whole-brain analysis uncovered other brain\nregions that share information with the PPA-RSC scene network.\n\n1 Introduction\nTo understand how the brain represents and processes information we must account for two com-\nplementary properties: information is represented in a distributed fashion, and brain regions are\nstrongly interconnected. Although heralded as a tool to address these issues, functional magnetic\nresonance imaging (fMRI) initially fell short of achieving these goals because of limitations of tra-\nditional analysis methods, which treat voxels as independent. Multi-voxel pattern analysis (MVPA)\nhas revolutionized fMRI analysis by accounting for distributed patterns of activity rather than abso-\nlute activation levels. The analysis of functional connectivity, however, is so far mostly limited to\ncomparing the time courses of individual voxels. To overcome these limitations we demonstrate a\nnew method of pattern-based functional connectivity analysis based on mutual information of sets\nof voxels. Furthermore, we show that selecting voxels based on the mutual information of local\nactivity with respect to ground truth outperforms other voxel selection methods.\nWe apply our new analysis methods to the decoding of natural scene categories from the human\nbrain. Human observers are able to quickly and ef\ufb01ciently perceive the content of natural scenes [15,\n26]. It was recently shown by [23] that activity patterns in the parahippocampal place area (PPA), the\nretrosplenial cortex (RSC), the lateral occipital complex (LOC), and, to some degree, primary visual\ncortex (V1) contain information about the categories of natural scenes. To truly understand how\nthe brain categorizes natural scenes, however, it is necessary to grasp the interactions between these\nregions of interest (ROIs). Our new technique for pattern-based functional connectivity enables\nus to uncover shared scene category-speci\ufb01c information among the ROIs. When con\ufb01gured for\nexploratory whole-brain analysis, the technique allows us to discover other brain regions that may\nbe involved in natural scene categorization.\nMutual information is appropriate for fMRI analysis if one considers fMRI data as a noisy com-\nmunication channel in the sense of Shannon\u2019s information theory [19]; the information contained\n\n\u2217Barry Chai and Dirk B. Walther contributed equally to this work.\n\u2020Diane M. Beck and Li Fei-Fei contributed equally to this work.\n\n1\n\n\fin a population of neurons must be communicated through hemodynamic changes and concomitant\nchanges in magnetization which can be measured as the blood-oxygen level dependent (BOLD)\nfMRI signal, then proceed through several layers of data processing, culminating in a single time\nvarying value in a particular voxel. While this noisy communication concept has been embraced\nby the brain machine interface community [25], information theory has, thus far, been less utilized\nin the fMRI analysis community (see [8] for exceptions). This may be partly due to the numerical\ndif\ufb01culties in estimating the probability distributions necessary for computing mutual information.\nThis problem is exacerbated when patterns of voxels are considered. In this case distributions of\nhigher dimensionality need to be estimated from preciously few data points. Recent developments\nin information theory, however, help us overcome these hurdles.\nIn Section 2 we review these theoretical advances and adapt them for our dual purpose of voxel\nselection and pattern-based functional connectivity analysis. Following a discussion of related work\nin Section 3, in Section 4 we apply our new methods to fMRI data from an experiment on distin-\nguishing natural scene categories in the human brain. We lay conclude the paper in Section 5.\n\n2 Multivariate mutual information for fMRI data\nInformation theory was originally formulated for discrete variables. In order to adapt the theory to\ncontinuous random variables, the underlying probability distribution needs to be estimated from the\nsampled data points. Previous work such as [7, 18] have used \ufb01xed bin-size histogram or Parzen\nwindow methods for this purpose. However, these methods do not generalize to high-dimensional\ndata. Recently, Perez-Cruz has shown that a k-nearest-neighbor (kNN) approach to estimating infor-\nmation theoretic measures converges to the true information theoretic measures asymptotically with\n\ufb01nite k, even in higher dimensional spaces [16]. In this section we adapt this strategy to estimate\nmulti-voxel mutual information.\n\n2.1 Nearest-neighbor mutual information estimate\nIn information theory, the randomness of a probability distribution is measured by its entropy. For a\ndiscrete random variable x, entropy can be calculated as\n\np(xi) log p(xi).\n\n(1)\n\ni=1\n\nMutual information is intuitively de\ufb01ned as the reduction of the entropy of the random variable x by\nthe entropy of x after y is known:\n\nI(x, y) = H(x) \u2212 H(x|y).\n\n(2)\n\nH(x) = \u2212 n(cid:88)\n\nThe separation into entropies allow us to calculate mutual information for multivariate data. Random\nvariables x, y can be of arbitrary dimensions.\nAs shown in [24], using kNN estimation, entropies and conditional entropies can be de\ufb01ned as\n\nn(cid:88)\nH(x) = \u2212 1\nn(cid:88)\nn\nH(x|y) = \u2212 1\nn\n\ni=1\n\nlog pk(xi),\n\n,\n\n(3)\n\n(4)\n\nlog pk(xi, yi)\npk(yi)\n\ni=1\n\nwhere the summation is over n data points, each represented by xi. pk(xi) is the kNN density\nestimated at xi. pk(xi) is de\ufb01ned as\n\npk(xi) = k\n\nn \u2212 1\n\n\u0393(d/2 + 1)\n\n\u03c0d/2\n\n1\n\nrk(xi)d .\n\n(5)\n\nwhere \u0393 is the gamma function, d is the dimensionality of xi and rk(xi) is the Euclidean distance\nfrom xi to the kth nearest training point. pk(xi) is the probability density function at xi, which is a\nset of voxel values for a given category task(or label) in the context of our fMRI experiment.\n\n2\n\n\ffMRI multivariate information analysis\n\n2.2\nIn previous work, such as [7], information theory has been used as a measure for functional connec-\ntivity of one voxel to another voxel. While such analysis is valuable for exploring connections in the\nbrain, it does not fully leverage the information stored in the local pattern of voxels. In this section\nwe propose a framework for multivariate information analysis of fMRI data for dual purposes: voxel\nselection and functional connectivity.\n\n2.2.1 Voxel selection based on mutual information with respect to ground truth label\nFor voxel selection we are interested in \ufb01nding a subset of voxels that are highly informative for\ndiscriminating between the ground truth labels in the experiment. This is a useful step that serves two\npurposes. From a machine learning perspective, reducing the dimensionality of the brain image data\ncan boost classi\ufb01er performance and reduce classi\ufb01er variance. From a neuroscience perspective,\nthe locations of highly informative voxels identify functional regions involved in the experiment. To\nachieve both of these goals we use a multivariate mutual information measure to analyze a localized\npattern of M voxels. This local analysis windows is moved across the brain image. At each location\nwe estimate the mutual information shared between the pattern of M voxels and the experiment label.\nIn our experiments we choose M = 7 to evaluate the smallest symmetrical pattern around a center\nvoxel, which consists of the center voxel and its 6 face-connected neighbors. Mutual information\nbetween voxels V and labels L is de\ufb01ned as\n\nUsing equation 1 the entropies can be calculated by\n\nI(V, L) = H(V ) + H(L) \u2212 H(V, L).\n\nn(cid:88)\n\ni=1\n\nn(cid:88)\n\ni=1\n\nI(V, L) = \u2212 1\nn\n\nlog pk(Vi) \u2212 1\nn\n\nlog pk(Li) +\n\n1\nn\n\n(6)\n\n(7)\n\nlog pk(Vi, Li),\n\nn(cid:88)\n\ni=1\n\nwhere n is the number of data-points observed, Li is the experiment label for ith data point, Vi is a\n7-dimensional random variable, Vi = (vi1,vi2,vi3,vi4,vi5,vi6, vi7) with each entry corresponding to\none of 7 voxels\u2019 values at data point i. Equation 7 can be used to compute the mutual information\nof localized set of voxels Vi with respect to their ground truth label Li. We can then perform voxel\nselection by selecting the locations of highest mutual information. This is useful as a preprocessing\nstep before applying any machine learning algorithms and as well as a way to spatially map out the\ninformative voxels with respect to the task.\n\n2.2.2 Functional connectivity by shared information between distributed voxel patterns\nTwo distributed brain regions can be modeled as a communication channel. Measuring the mutual\ninformation across the two regions provides an intuitive measure for their functional connectivity.\nThe voxel values observed in each region can be regarded as observed data from an underlying prob-\nability distribution \u2013 the distribution that characterizes the functional region under the experiment\ncondition.\nPrevious approaches have analyzed shared information in a univariate way, computing the mutual\ninformation between two voxels. However such univariate information analysis disregards the in-\nformation stored in the local patterns of voxels. In this work we present a multivariate information\nanalysis that estimates shared information between two sets of voxels that leverages the information\nstored in the local patterns:\n\nI(V, S|L) = H(V |L) + H(S|L) \u2212 H(V, S|L),\n\n(8)\n\nwhere V and S are random variables for sets of 7 voxels. L is the experiment label. Using equations\n3 and 4 this can be written as\n\nn(cid:88)\n\ni=1\n\nn(cid:88)\n\ni=1\n\nn(cid:88)\n\ni=1\n\nI(V, S|L) = \u2212 1\nn\n\nlog pk(Vi, Li)\npk(Li)\n\n\u2212 1\nn\n\nlog pk(Si, Li)\npk(Li)\n\n+\n\n1\nn\n\nlog pk(Vi, Si, Li)\n\npk(Li)\n\n.\n\n(9)\n\nEquation 9 allows us to measure the functional connectivity between two distributed sets of voxels\nV and S by computing the mutual information between the two sets of voxels conditioned on the\nexperiment task label L. We show in our experiments (sec 4.4) that by using this measurement, our\nalgorithm can uncover meaningful functional connectivity patterns among regions of the brain.\n\n3\n\n\fFigure 1: Comparison of decoding accuracy3 between MI voxel selection and other standard voxel selection\nmethods(refer to section 4.2). The single voxel MI approach surpasses most discr.1 voxel selection but performs\non par with most active2 voxel selection. Using a pattern of 7 voxels, the MI7D approach achieves the highest\ndecoding accuracy. At 600 voxels, MI7D decoding accuracy3 is signi\ufb01cantly higher than most active with p-\nvalue < 0.05. At 1250 voxels, MI7D decoding accuracy is signi\ufb01cantly higher than MI1D with p-value < 0.01\n(This \ufb01gure must be viewed in color)\n\n3 Related work\nStatistical relationships between different parts of the brain, referred to as functional connectivity,\nhave been computed with a number of different methods. The methods can be broadly classi\ufb01ed as\neither data-driven or model-based [10].\nIn data-driven approaches, no speci\ufb01c hypothesis of connectivity is used, but large networks of\nbrain regions are discovered based purely on the data. Most commonly, this is achieved with a\ndimensionality-reduction procedure such as principal component analysis (PCA) or independent\ncomponent analysis (ICA). Originally applied to the analysis of PET data [5], PCA has also been\napplied to fMRI data (see [12]). ICA has been gained interest for the investigation of the so-called\ndefault network in the brain at rest [11].\nModel-based approaches test a prior hypothesis about the statistical relations between a seed voxel\nand a target voxel. By \ufb01xing the seed voxel and moving the target voxel all over the brain, a\nconnectivity map with respect to the seed voxel can be generated. The statistical relation of the\ntwo voxels is usually modeled assuming temporal dependence between voxels in methods such as:\ncross-correlation [2], coherence [21], Granger causality [1], or transfer entropy [20].\nThese methods compare the time courses of individual voxels. Following the same principal idea,\nwe model functional connectivity based on the mutual information between sets of seed and target\nvoxels to leverage the spatial information contained in activity patterns among voxels rather than\nthe temporal information between two voxels. fMRI has a higher spatial resolution than tempo-\nral resolution. We design our mutual information connectivity measure to exploit this property of\nfMRI data. Yao et al. [26] have also explored pattern-based functional connectivity by modeling\nthe interactions between distributed sets of voxels with a generative model. We take a simpler ap-\nproach by using only the multivariate information measure which allows us to explore for unknown\nconnections in the whole brain in a searchlight manner.\nIn recent years it has become apparent that patterns of fMRI activity hold more detailed information\nabout experimental conditions than the activation levels of individual voxels [6].\nIt is therefore\n\n1Most discri. \u2013 Most discriminative voxels are those showing the largest difference in activity between any\n\npair of scene categories.\n\n2Most active \u2013 Most active voxels are those showing the largest difference in activity between the \ufb01xation\n\ncondition and viewing images of any category.\n\n4\n\n020040060080010001200140000.10.30.40.5Number of voxelsDecoding accuracy(cid:7716)(cid:7716)most discr.randomchance level16Mut. Info. (7 vox.)Mut. Info. (1 vox.)most active\fFigure 2: Locations of voxels with high 7D mutual information with respect to scene category label. The\nknown functional areas that respond to scenes and visual stimuli such as PPA, RSC, V1 are all selected, which\nalso explains the high decoding accuracy using the selected voxels. The brain maps shown above are based on\ngroup analysis over 5 subjects superimposed on an MNI standard brain. (This \ufb01gure must be viewed in color)\n\ncogent to also consider the information contained in voxel patterns for the analysis of functional\nconnectivity. We achieve this by computing the mutual information of a pattern of locally connected\nvoxels at the seed location with a pattern at the target location. As with the univariate functional\nconnectivity analysis, this multivariate version also allows us to test hypotheses about connectivity\nof brain regions as well as generate connectivity maps.\nBecause of the large number of voxels in the brain (many thousands, depending on resolution),\nmultivariate techniques usually require some kind of feature selection or dimensionality reduction.\nThis can be achieved by focusing on pre-de\ufb01ned ROIs, or by selecting voxels form the brain based\non some statistical criteria [3, 14]. Here we show that using mutual information of individual voxels\nwith respect to ground truth for voxel selection works at the same level as these previous methods,\nbut that mutual information of patterns of voxels with respect to ground truth outperforms all of the\nunivariate methods we tested.\nInformation theory has been applied to fMRI data in the context of brain machine interfaces [25],\nto generate activation maps [8], for effective connectivity in patients [7], and for image registration\n[17]. However, to our knowledge this is the \ufb01rst application to both voxel selection and functional\nconnectivity based on multivariate activity patterns.\n4 Experiments\n4.1 Data\nFor the experiments described in this section we use the data from the fMRI experiment on natural\nscene categories by [23]. Brie\ufb02y, \ufb01ve participants passively viewed color images belonging to six\ncategories of natural scenes (beaches, buildings, forests, highways, industry, and mountains). Stim-\nuli were arranged into blocks of 10 images from the same natural scene category. Each image was\ndisplayed sequentially for 1.6 seconds. A run was composed of 6 blocks, one for each natural scene\ncategory, interleaved with 12 s \ufb01xation periods. Images were presented upright inverted on alter-\nnating runs, with each inverted run preserving the image and category order used in the preceding\nupright run. A session contained 12 such runs, and the order of categories was randomized across\nblocks. Each subject performed two blocks with a total of 24 runs. In total we have 1192 data\npoints per subject across all 6 categories. The data obtained from the authors in [23] contains only\nlocalizers for V1, PPA, RSC, LOC, FFA areas. Thus we limit our seed areas to these ROIs.\n4.2 Voxel selection\nThe goal of voxel selection is to identify the most relevant voxels for the experiment task out of the\ntens of thousands of voxels in the entire brain. A quantitative evaluation of voxel selection is the\ndecoding accuracy3 of the selected voxels, which measures how well can the selected voxels predict\nthe viewing condition from the neural responses.\nFig.1 compares our mutual information-based voxel selection method to other voxel selection meth-\nods. Decoding accuracy3 using univariate kNN mutual information is comparable to most active1\n\n5\n\n00.17MILR\f(a) Comparing 7D and 1D mutual information within-ROI\nconnections\n\n(b) 7D MI between-ROI connections\n\nFigure 3: a) Within-ROI MI values for 7D and 1D mutual information, b) Schematic showing the signi\ufb01cant\nROI connections found using 7D mutual information analysis. The network shows strong connections between\nPPA and RSC, both ipsilaterally and contralaterally. \u2217\u2217p < 10\u22126, \u2217p < 0.01\n\nvoxel selection. Multivariate information measure is able to select the more informative voxels by\nconsidering a local pattern of voxels jointly, leading to a boost in decoding accuracy3.\nTo further understand why multivariate mutual information boosts the decoding accuracy3, we can\nlook at the spatial locations of the informative voxels selected by multivariate information analysis\nshown in Fig.2. The most informative voxels selected correspond to known functional regions for\nscenes. In this \ufb01gure we see the scene areas V1, RSC, PPA, LOC that were also identi\ufb01ed in [23].\nInterestingly, our automatic voxel selection achieves a higher decoding accuracy3 than the ROIs\nselected by localizer in [23]. This may suggest that the multivariate information voxel selection is a\nbetter segmentation of the relevant ROIs than the localizer runs.\n\n4.3 Functional connectivity of ROIs\n\nIn the previous section, we have shown that multivariate information can effectively select infor-\nmative voxels for classi\ufb01cation.\nIn this section, we \ufb01rst illustrate the increased sensitivity of a\nmultivariate assessment of functional connectivity within known ROIs. Then we use multivariate\ninformation to explore connections between ROIs.\nA good comparison for the functional connectivity measure is the within-ROI connectivity. Voxels\nwithin the same ROI should exhibit high functional connectivity with each other. In Fig.3a we com-\npared our 7D measures with equivalent one dimensional measures using within-ROI connectivity.\nTo this end we randomly selected 15 seed and 15 target locations within each ROI, making sure that\nseed and target patterns have no voxels in common. Then we computed mutual information between\nall seed and all target locations, either using individual voxels (1D case) or patterns of seven voxels\n(7D case). Fig.3a shows the mean of the mutual information values for these two cases in each ROI.\nIn all ROIs, we \ufb01nd that multivariate information measure(7D) is signi\ufb01cantly higher than the uni-\nvariate measure(1D), suggesting that a pattern-based mutual information has a higher \ufb01delity than\nunivariate-based mutual information in mapping out functional connections.\nAfter having established that 7D mutual information signi\ufb01cantly outperforms 1D mutual informa-\ntion we proceed to calculate the between-ROI connectivity for scene areas V1, left/right PPA, and\nleft/right RSC using 7D mutual information as shown in Fig.3b. Between-ROI connectivity is de-\n\n3Decoding accuracy is obtained with a leave-two-runs-out cross-validation on the our scene data. In each\nfold two runs from viewing the same images upright and inverted are left out as test data. Voxel selection is\nperformed on the training runs using k = n/2, where n is the number of training examples in each category.\nUsing selected voxels, a linear SVM classi\ufb01er is trained on the upright runs with C = 0.02 as in [23]. In testing\nwe use majority voting on the SVM prediction labels to vote for the most likely scene label for each block of\ndata. Decoding accuracy is the average of cross-validation accuracy over the 5 subjects.\n\n6\n\nlPPArPPAlRSCrRSC00.511.52Log  (6)3MI(cid:7716)MI 7DMI 1D*2*******lPPArPPAlRSCrRSCV10.95** 0.84** 1.1** 0.71** 0.85** 1.16** 0.75* \fFigure 4: Connectivity map seeding from left PPA. Talairach coordinates de\ufb01ned in [22] are shown as the Z\nand Y coordinates for axial and coronal slices respectively. The intensity of the maps shows the MI values(This\n\ufb01gure must be viewed in color)\n\nFigure 5: Overlap analysis showing areas where overlap occurs with the strongest connections from more than\ntwo scene network ROIs. Talairach coordinates de\ufb01ned in [22] are shown as the Z and Y coordinates for axial\nand coronal slices respectively. The color code indicates the amount of overlap.(This \ufb01gure must be viewed in\ncolor)\n\n\ufb01ned similarly as within-ROI connectivity except that seed and target locations are the chosen in\ndifferent ROIs.\nA number of aspects of the connections mapped out with MI7D analysis agree with neuroscience\n\ufb01ndings. First, it is expected that PPA and RSC should be strongly connected as part of a scene\nnetwork. Moreover, since V1 is the input to the cortical visual system, it is also likely that it should\nshare information with at least one member of the scene network, which in this case was the right\nPPA. One novel \ufb01nding from this analysis is that all of the strongest connections we discovered\nincluded right RSC. In particular, right RSC shares strong connections with left RSC, right PPA and\nleft PPA, suggesting that right RSC may play a particularly important role in distinguishing natural\nscene categories. More work will be needed to verify this hypothesis.\nTo summarize, we have veri\ufb01ed that multivariate information analysis can reliably map out con-\nnections within and between ROIs known to be involved in processing natural scene categories. In\nthe next section we show how the same analysis can be extended to uncover other ROIs that share\ninformation with this scene network in our scene classi\ufb01cation experiment.\n\n4.4 Functional connectivity - whole brain analysis\nWhile it is valuable to con\ufb01rm existing hypotheses about areas that represent scene categories, it\nis also interesting to uncover new brain areas that might be related to scene categorization. In this\nsection, we show that we can use our multivariate information analysis approach to explore other\nareas outside of the known ROIs that form strong connections with the known ROIs.\nFor each of the functional areas in the scene network, we can explore other areas connected to it.\nAs in section 3 we measure functional connectivity as multivariate mutual information between the\nseed and candidate target areas. We \ufb01x the seed area to an ROI de\ufb01ned by a localizer. The candidate\narea moves around the brain, at each location measuring the mutual information with respect to the\nseed area.\n\n7\n\n0.370.51rRSCleft PrecuneuslPPArPPAlRSCrRSCleft Medial Frontal GyruslPPArPPAright Inferior Frontal GyrusMIZ = -13 mmZ = 23 mmY = 52 mmLRleft Medial Frontal GyruslRSCrRSClPPArPPAleft CuneusZ = 26 mmY = 43 mmZ = 29 mmleft Medial Frontal Gyrusleft Cuneusright Cuneusright Precuneus- 4 overlap- 3 overlap- 2 ovlerapLR\f4.4.1 Con\ufb01rming known connections\nFig.4 shows an example of the connectivity map seeding from left PPA. Each highlighted location in\nthe connectivity map shows its connectivity to left PPA as measured by the multivariate information.\nAs shown in Fig.4, both left and right PPA are highlighted, con\ufb01rming their bilateral connection.\nFurthermore, we see strong connections between left PPA and left and right RSC. A minimum\ncluster size of 13 is used to threshold the connectivity map. The minimum cluster size is determined\nby AlphaSim in AFNI [4]. Notice in Fig.4 that the highest MI in the whole-brain analysis has MI of\n0.51 whereas the within-ROI MI of left PPA in Fig.3b has a value of 1.5. The decrease in MI is due\nto the smoothing of connectivity maps when we combine them across subjects.\n4.4.2 Discovering new connections\nBesides con\ufb01rming known regions of the scene network, our connectivity maps allow us to explore\nother brain areas that might be related to the scene network. In Fig.4 we not only observe known\nscene network ROIs but additional areas such as the right Inferior Frontal Gyrus, left Medial Frontal\nGyrus, and left Precuneus. Interestingly, the Inferior Frontal Gyrus, typically associated with lan-\nguage processing [13], also showed up in a searchlight analysis for decoding accuracy in [23].\nSo far we have examined how the rest of the brain connects to one ROI in the scene network,\nspeci\ufb01cally we used left PPA as the example. However, to further strongly establish which regions\nare functionally connected in regards to distinguishing scene category, we asked which brain areas\nare strongly connected to two or more of the scene network ROIs. Areas that connect to more than\none of the scene network ROIs are particularly interesting, because having multiple connections\nstrengthens evidence that they play a signi\ufb01cant role in distinguishing scene categories.\nTo investigate this question, we generate one connectivity map for each of the 4 scene network ROIs,\nsimilar to Fig.4. We take the areas with the top 5 percent highest mutual information in each of the\n4 maps and overlap them. Fig.5 shows this overlap analysis.\nSimilar to the previous analysis, the overlap analysis highlights all 4 known areas of the scene\nnetwork. Interestingly, this analysis shows that right RSC and right PPA are connected with more\nregions of the scene network than left RSC and PPA. This suggests that perhaps there is a laterality\neffect in the scene network that could be investigated in future studies.\nFurthermore, we can also explore areas outside of the scene network with the overlap analysis.\nIn Fig.5, left/right Cuneus and right Precuneus, highlighted in orange, exhibit strong connections\nwith 3/4 of the scene network ROIs. Left Medial Frontal Gyrus is strongly connected to 2/4 of the\nscene network ROIs. These exploratory areas also point to interesting future investigations for scene\ncategory studies.\n5 Conclusion\nIn this paper we have introduced a new method for evaluating the mutual information that patterns\nof fMRI voxels share with the ground truth labels of the experiment and with patterns of voxels\nelsewhere in the brain. When used as a voxel selection method for subsequent decoding of viewed\nnatural scene category, mutual information of patterns of voxels with respect to the ground truth\nlabel is superior to mutual information of individual voxels.\nWe have shown that mutual information of voxel patterns in two ROIs is a more sensitive measure\nof task-speci\ufb01c functional connectivity analysis than mutual information of individual voxels. We\nhave identi\ufb01ed a network of regions consisting of left and right PPA and left and right RSC that\nshare information about the category of a natural scene viewed by the subject. Connectivity maps\ngenerated with this method have identi\ufb01ed left medial frontal gyrus, left/right cuenus, and right\nprecuneus as sharing scene-speci\ufb01c information with PPA and RSC. This could stimulate interesting\nfuture work such as estimating mutual information for an even larger set of voxels and understanding\nthe exploratory areas highlighted by this analysis. Although we con\ufb01ned our experiments to data\nfrom a scene category task, all the analysis proposed here could be used for other tasks in other\ndomains.\nAcknoledgements\nThis work is funded by National Institutes of Health Grant 1 R01 EY019429 (to L.F.-F., D.M.B., D.B.W.), a\nBeckman Postdoctoral Fellowship (to D.B.W.), a Microsoft Research New Faculty Fellowship (to L.F.-F.), and\nthe Frank Moss Gift Fund (to L.F-F.). The authors would like to thank Todd Coleman and Fernando Perez-Cruz\nfor the helpful discussions on entropy estimation.\n\n8\n\n\fReferences\n\n[1] Granger, C. W. J.\n\nInvestigating causal relations by econometric models and cross-spectral methods.\n\nEconometrica 37, 424-438, 1969\n\n[2] J. Cao and K. Worsley The geometry of correlation \ufb01elds with an application to functional connectivity\n\nof the brain. Ann. Appl. Probab, 9:1021C1057, 1998.\n\n[3] D. Cox and R. Savoy Functional magnetic resonance imaging (fMRI) \u201cbrain reading\u201d: Detecting and\nclassifying distributed patterns of fMRI activity in human visual cortex. NeuroImage, 19(2):261C270,\n2003.\n\n[4] RW Cox. AFNI: Software for analysis and visualization of functional magnetic resonance neuroimages.\n\nComputers and Biomedical Research, 29:162-173, 1996.\n\n[5] K. J. Friston, C. D. Frith, P. F. Liddle, and R. S. Frackowiak. Functional connectivity: the principal-\n\ncomponent analysis of large (PET) data sets. J Cereb Blood Flow Metab, 13(1):5C14, January 1993.\n\n[6] J. V. Haxby, M. I. Gobbini, M. L. Furey, A. Ishai, J. L. Schouten, and P. Pietrini. Distributed and overlap-\nping representations of faces and objects in ventral temporal cortex. Science, 293(5539):2425C30, 2001.\nJournal Article United States.\n\n[7] H. Hinrichs, H. Heinze, and M. Schoenfeld. Causal visual interactions as revealed by an information\n\ntheoretic measure and fMRI. NeuroImage, 31(3):1051 C 1060, 2006.\n\n[8] A. T. John, J. W. F. Iii, W. M. W. Iii, J. Kim, and A. S.\u2018Willsky. Analysis of functional MRI data using\n\nmutual information, 1999.\n\n[9] Fei-Fei, L., Iyer, A., Koch, C., Perona, P. (2007). What do we perceive in a glance of a real-world scene?\n\nJournal of Vision,7(1):10, 1-29, http://journalofvision.org/7/1/10/,doi:10.1167/7.1.10.\n\n[10] K. Li, L. Guo, J. Nie, G. Li, and T. Liu. Review of methods for functional brain connectivity detection\nusing fMRI. Computerized medical imaging and graphics: The Of\ufb01cial Journal of the Computerized\nMedical Imaging Society, 33(2):131C139, March 2009.\n\n[11] M. J. Mckeown, S. Makeig, G. G. Brown, T.-P. Jung, S. S. Kindermann, R. S. Kindermann, A. J. Bell,\nand T. J. Sejnowski. Analysis of fMRI data by blind separation into independent spatial components.\nHuman Brain Mapping, 6:160C188, 1998.\n\n[12] A. Meyer-Baese, A. Wismueller, and O. Lange. Comparison of two exploratory data analysis methods\nfor fMRI: unsupervised clustering versus independent component analysis. Information Technology in\nBiomedicine, IEEE Transactions on, 8(3):387C398, Sept. 2004.\n\n[13] Geschwind N. (1970) The organization of language and the brain. Science 170:940 944..\n[14] D. Neill, A. Moore, F. Pereira, and T. Mitchell. Detecting signi\ufb01cant multidimensional spatial clusters.\n\nIn Proceedings of Neural Information Processing Systems, 2004.\n\n[15] M. Potter. Short-term conceptual memory for pictures. Journal of Experimental Psychology: Human\n\nLearning and Memory, 2(5):509C522, 1976.\n\n[16] F. Perez-Cruz. Estimation of information theoretic measures for continuous random variables. In D.\n\nKoller, D. Schuurmans, Y. Bengio, and L. Bottou, editors, NIPS, pages 1257C1264. MIT Press, 2008.\n\n[17] Pluim, J. P. W. and Maintz, J. B. A. and Viergever, M. A. Mutual-information based registration of\n\nmedical images: a survey. IEEE Trans Med Imaging, 2003, 22:986C1004.\n\n[18] X. U. Rui, C. H. E. N. Yen-Wei, T. A. N. G. Song-Yuan, S. Morikawa, and Y. Kurumi. Parzen-window\nbased normalized mutual information for medical image registration. IEICE Transactions on Informa-\ntion and Systems, E91-D(1):132C144, January 2008.\n\n[19] C. E. Shannon. A Mathematical Theory of Communication. CSLI Publications, 1948.\n[20] T. Schreiber. Measuring Information Transfer. Physical Review Letters, vol. 85, no. 2, pp. 461+, July\n\n2000.\n\n[21] F. T. Sun, L. M. Miller, and M. DEsposito. Measuring interregional functional connectivity using coher-\n\nence and partial coherence analyses of fmri data. NeuroImage, 21(2):647 C 658, 2004.\n\n[22] J. Talairach and P. Tournoux. Co-planar Stereotaxic Atlas of the Human Brain: 3-Dimensional Propor-\n\ntional System - an Approach to Cerebral Imaging. Thieme Medical Publishers, New York, 1988\n\n[23] D. Walther, E. Caddigan, L. Fei-Fei*, and D. Beck* (2009), Natural scene categories revealed in\ndistributed patterns of activity in the human brain. The Journal of Neuroscience, 29(34):10573-\n10581.(*indicates equal contribution)\n\n[24] Q. Wang, S. Kulkarni, and S. Verdu. Divergence estimation of continuous distributions based on data-\n\ndependent partitions. Information Theory, IEEE Transactions on, 51(9):3064C3074, Sept. 2005.\n\n[25] B. D. Ward and Y. Mazaheri. Information transfer rate in fMRI experiments measured using mutual\ninformation theory. Journal of Neuroscience Methods, 167(1):22 C 30, 2008. Brain-Computer Interfaces\n(BCIs).\n\n[26] B. Yao, D.B. Walther, D.M. Beck*, L. Fei-Fei*. Hierarchical Mixture of Classi\ufb01cation Experts Uncovers\n\nInteractions between Brain Regions. NIPS, 2009. (* indicates equal contribution)\n\n9\n\n\f", "award": [], "sourceid": 1140, "authors": [{"given_name": "Barry", "family_name": "Chai", "institution": null}, {"given_name": "Dirk", "family_name": "Walther", "institution": null}, {"given_name": "Diane", "family_name": "Beck", "institution": null}, {"given_name": "Li", "family_name": "Fei-fei", "institution": null}]}