{"title": "Generalisation of structural knowledge in the hippocampal-entorhinal system", "book": "Advances in Neural Information Processing Systems", "page_first": 8484, "page_last": 8495, "abstract": "A central problem to understanding intelligence is the concept of generalisation. This allows previously learnt structure to be exploited to solve tasks in novel situations differing in their particularities. We take inspiration from neuroscience, specifically the hippocampal-entorhinal system known to be important for generalisation. We propose that to generalise structural knowledge, the representations of the structure of the world, i.e. how entities in the world relate to each other, need to be separated from representations of the entities themselves. We show, under these principles, artificial neural networks embedded with hierarchy and fast Hebbian memory, can learn the statistics of memories and generalise structural knowledge. Spatial neuronal representations mirroring those found in the brain emerge, suggesting spatial cognition is an instance of more general organising principles. We further unify many entorhinal cell types as basis functions for constructing transition graphs, and show these representations effectively utilise memories. We experimentally support model assumptions, showing a preserved relationship between entorhinal grid and hippocampal place cells across environments.", "full_text": "Generalisation of structural knowledge in the\n\nhippocampal-entorhinal system\n\nJames C.R. Whittington*\nUniversity of Oxford, UK\n\njames.whittington@magd.ox.ac.uk\n\nTimothy H. Muller*\n\nUniversity of Oxford, UK\n\ntimothymuller127@gmail.com\n\nShirley Mark\n\nUniversity College London, UK\n\ns.mark@ucl.ac.uk\n\nCaswell Barry\n\nUniversity College London, UK\ncaswell.barry@ucl.ac.uk\n\nTimothy E.J. Behrens\nUniversity of Oxford, UK\nbehrens@fmrib.ox.ac.uk\n\nAbstract\n\nA central problem to understanding intelligence is the concept of generalisation.\nThis allows previously learnt structure to be exploited to solve tasks in novel sit-\nuations differing in their particularities. We take inspiration from neuroscience,\nspeci\ufb01cally the hippocampal-entorhinal system known to be important for generali-\nsation. We propose that to generalise structural knowledge, the representations of\nthe structure of the world, i.e. how entities in the world relate to each other, need to\nbe separated from representations of the entities themselves. We show, under these\nprinciples, arti\ufb01cial neural networks embedded with hierarchy and fast Hebbian\nmemory, can learn the statistics of memories and generalise structural knowl-\nedge. Spatial neuronal representations mirroring those found in the brain emerge,\nsuggesting spatial cognition is an instance of more general organising principles.\nWe further unify many entorhinal cell types as basis functions for constructing\ntransition graphs, and show these representations effectively utilise memories.\nWe experimentally support model assumptions, showing a preserved relationship\nbetween entorhinal grid and hippocampal place cells across environments.\n\n1\n\nIntroduction\n\nAnimals have a remarkable ability to \ufb02exibly take knowledge from one domain and generalise it to\nanother. This is not yet the case for machines. The advantages of generalising knowledge are clear\n- it allows one to make quick inferences in new situations, without having to always learn afresh.\nGeneralisation of statistical structure (the relationships between objects in the world) imbues an agent\nwith the ability to \ufb01t things/concepts together that share the same statistical structure, but differ in the\nparticularities, e.g. when one hears a new story, they can \ufb01t it in with what they already know about\nstories in general, such as there is a beginning, middle and end - when the funny story appears while\nlistening to the the news, it can be inferred that the programme is about to end.\nGeneralisation is a topic of much interest. Advances in machine learning and arti\ufb01cial intelligence\n(AI) have been impressive [23, 28], however there is scepticism over whether \u2019true\u2019 underlying\nstructure is being learned. We propose that in order to learn and generalise structural knowledge, this\nstructure must be represented explicitly, i.e. separated from the representations of sensory objects in\n\n32nd Conference on Neural Information Processing Systems (NeurIPS 2018), Montr\u00e9al, Canada.\n\n\fthe world. In worlds that share the same structure but differ in sensory objects, explicitly represented\nstructure can be combined with sensory information in a conjunctive code unique to each environment.\nThus sensory observations are \ufb01t with prior learned structural knowledge, leading to generalisation.\nIn order to understand how we may construct such a system, we take inspiration from neuroscience.\nThe hippocampus is known to be important for generalisation, memory, problems of causality, infer-\nential reasoning, transitive reasoning, conceptual knowledge representation, one-shot imagination,\nand navigation [13, 7, 17, 25]. We propose the statistics of memories in hippocampus are extracted by\ncortex [27], and future hippocampal representations/memories are constrained to be consistent with\nthe learned structural knowledge. We \ufb01nd this an interesting system to model using arti\ufb01cial neural\nnetworks (ANNs), as it may offer insights into generalisation for machines, further our understanding\nof the biological system itself, and continue to link neuroscience and AI research [18, 38].\n\n1.1 Background\n\nIn spatial navigation there is a good understanding of neuronal representations in both hippocampus\n(e.g. place, landmark cells) and medial entorhinal cortex (e.g. grid, border, object vector cells). Thus\nwhen modelling this system, we start with problems akin to navigation so we can both leverage and\ncompare our results to these known representations (noting our approach is more general). Place [29]\nand grid cells [16] have had a radical impact in neuroscience, leading to the 2014 Nobel Prize in\nPhysiology and Medicine. Place and grid cells are similar in that they have a stable \ufb01ring pattern\nfor speci\ufb01c regions of space. Place cells tend to only \ufb01re in a single (or couple) location in a given\nenvironment, whereas grids cells \ufb01re in a regular lattice pattern tiling the space. These cells cemented\nthe idea of a \u2019cognitive map\u2019, where an animal holds an internal representation of the space it navigates\n[36]. Traditionally these cells were believed to be spatial only. It has since emerged that place and\ngrid cells code for both spatial and entirely non-spatial dimensions such as sound frequency [2], and\nfurthermore grid-like codes for two dimensional (2D) non-spatial coordinate systems exist [9]. It\ntherefore seems that place and grid codes may provide a general way of representing information.\nOther entorhinal cell types (border [31], object vector cells [19]) appear to have disparate roles in\ncoding space. Here we unify them, along with grid cells, as basis functions for transition statistics.\nGrid cells may offer a generalisable structural code. Indeed grid cell representations are similar in\nenvironments that share structure ([14], section 5). Recent results suggest such codes summarise\nstatistics of 2D space, either via a PCA of hippocampal place cells [12] or as eigenvectors of the\nsuccessor representation [32]. These summary statistics represent rules of 2D-ness (not just \u2019spatial\u2019\nspace), e.g. if A is close to B, and B is close to C, we can infer A and C are close. Place cells\nmay offer a conjunctive representation. Their activity is modulated by the sensory environment as\nwell as location [39, 22]. Additionally the place cell code is different for two structurally identical\nenvironments - this is called remapping [6, 26]. Remapping is traditionally thought to be random.\nHowever, we propose place cells form a conjunctive representation between structural (grid cells)\nand sensory input, and therefore remap to non-random locations consistent with this conjunction.\n\n1.2 Contributions\n\nWe implement our proposal in an ANN tasked with predicting sensory observations when walking\non 2D graph worlds, where each vertex is associated with a sensory experience. To make accurate\npredictions, the agent should learn the underlying hidden structure of the graphs. We separate structure\nfrom sensory identity, proposing grid cells encode structure, and place cells form a conjunctive\nrepresentation between sensory identity and structure (Fig 1a). This conjunctive representation forms\na Hebbian memory, which bridges structure and identity, allowing the same structural code to be\nreused across environments that share statistics but differ in sensory experiences. We combine fast\nHebbian learning of episodic memories, with gradient descent which slowly learns to extract statistics\nof these memories. Our network learns representations that mirror those found in the brain, with\ndifferent entorhinal-like representations forming depending on transition statistics. We further present\nanalyses of a remapping experiment [5], which support our model assumptions, showing place cells\nremap to locations consistent with a grid code, i.e. not randomly as previously thought.\nThe key conceptual novelties are as follows. Neuroscience: We found an interpretation of grid cells,\nplace cells and remapping that offers a mechanistic understanding for the hippocampal involvement in\ngeneralisation of knowledge across domains. We provide a unifying framework for many entorhinal\n\n2\n\n\f(a) Conjunction\n\n(b) Schematic of task and model approach\n\nFigure 1: (a) Separated structural and sensory representations combined in a conjunctive code.\nLEC/MEC: Lateral/Medial entorhinal cortex, HPC: Hippocampus. (b) Problem the model faces -\nextracting generalisable statistics across domains, while rapidly learning the map within domain.\n\ncell types (grid cells, border cells, object vector cells) as building basis functions for transitions\nstatistics. Our results suggest spatial representations found in the brain may be an instance of a more\ngeneral coding mechanism organising knowledge across multiple domains. Machine learning: We\nhave built a network where fast Hebbian learning interacts with slow statistical learning. This allows\nus to learn representations whereby memories are not only stored in a Hebbian network for one-shot\nretrieval within domain, but also bene\ufb01t from statistical knowledge that is shared across domains -\nallowing zero shot inference.\n\n2 Related work\n\nConcurrently developed papers discovered grid-like and/or place-like representations in ANNs [4, 10].\nNeither paper uses memory or explains place cell phenomena. Both, however, use supervised learning\nin order to discover these representations, either supervising on actual x, y coordinates [10] or\nground truth place cells [4]. We use unsupervised learning, providing the network with only sensory\nobservations and actions. This is information available to a biological agent, unlike ground truth\nspatial representations. We further propose a role for grid cells in generalisation, not just navigation.\nOur modelling approach is simliar to [15? ]. However, we choose our memory storage and addressing\nto be computationally biologically plausible (rather than using other types of differentiable memory\nmore akin to RAM), as well as using hierarchical processing. This enables our model to discover\nrepresentations that are useful for both navigation and addressing memories. We also are explicit in\nseparating out the abstract structure of the space from any speci\ufb01c content (Fig 1a).\nWe follow a similar ideology to complementray learning systems [27] where the statistics of memories\nin hippocampus are extracted by cortex. We additionally propose that this learnt structural knowledge\nconstrains hippocampal representations in new contexts, allowing reuse of learnt knowledge.\n\n3 Model\n\nWe consider an agent passively moving on a 2D graph (Fig 1b), observing a non-unique sensory\nstimulus (e.g. an image) on each vertex. If the agent wishes to \u2019understand\u2019 its environment then it\nshould maximise its model\u2019s probability of observing each stimulus. The agent is trained on many\nenvironments sharing the same structure, i.e. 2D graphs, however the stimulus distribution is different\n(each vertex is randomly assigned a stimulus). There are various approaches to this problem, however\na generalisable one should exploit the underlying structure of the task - the 2D-ness of the space. One\nsuch approach is to have an abstract representation of space encoding relative locations, and then to\nplace a memory of what stimulus was observed at that (relative) location. Since the agent understands\nwhere it is in space, this allows for accurate state predictions to previously visited nodes even if the\nagent has never travelled along that particular edge before (e.g. loop closure in Figs 1b pink and 2c).\nAlthough we consider 2D graphs to compare learned representations to those found in the brain, our\napproach is appropriate for generalising other stuctural/relational/conceptual knowledge [25].\n\n3\n\nStructural code (grid cells / MEC)Sensory stimuli (LEC)Conjunctive code (place cells / HPC)Environments\u2026\u2026\u2026Sensory obsevationsHebbian learning rapidly remembers conjunction of location and sensory stimuli, allowing one-shot learningSGD slowly learns shared statistical knowledge across domains, allowing zero-shot inferenceTrials\f(a) Memory storage\n\n(b) Memory retrieval\n\n(c) Learning generalisable statistics\n\nFigure 2: Learning good representations. Small/large circles: high/low frequency cells. Grid cells\n(MEC) need to create (a) conjunctive Hebbian memories (in HPC, weights Mt between place cells)\nsuch that the same grid code can reinstate (b) the same memory via attractor dynamics. Grid cells are\nrecurrent (c), and so must learn transition weights such that they have the same code when returning\nto a state (loop closure). This code must be general enough to work across many environments.\n\nWe propose grid cells as bases for constructing abstract representations of space, and place cell\nrepresentations for the formation of fast episodic memories (Fig 2a). To link a stimulus to a given\n(relative) location, a memory should be a conjunction of abstract (relative) location and sensory\nstimulus, thus we propose place cells form a conjunctive representation between the sensorium and\ngrid input (Figs 1a and 2a). This is consistent with experimental evidence [39, 22]. We posit that\nthis is done hierarchically across spatial frequencies, such that the higher frequency statistics can\nbe repeatedly used across space. This reduces the number of weights that need to be learnt. This\nproposition is consistent with the hierarchical scales observed across both grid cells [34] and place\ncells [21], and with the entorhinal cortex receiving sensory information in hierarchical temporal\nscales [37]. We consider grid cells to be recurrent through time, allowing predictive state transitions\nto occur via grid cells (Fig 2c). This is consistent with grid codes being a natural basis for navigation\nin 2D spaces [33, 8].\nWe view the hippocampal-entorhinal system as one that performs inference. Grid cells make\ninferences based on their previous estimate of location in abstract space (and optionally sensory\ninformation linked to previous locations via memory). Place cells, a conjunction between the sensory\ndata and location in abstract space, are stored as memories. We consider sensory data, the item/object\nexperience of a state, as coming from the \u2019what stream\u2019 via lateral entorhinal cortex. The grid cells in\nour model, are the \u2019where stream\u2019 coming from medial entorhinal cortex (Fig 2). Our hippocampal\nconjunctive memory links \u2019what\u2019 to \u2019where\u2019, such that when we revisit \u2019where\u2019 we remember \u2019what\u2019.\n\n3.1 Model summary\n\nThe model is a neural network and learns structure across tasks. We optimise end-to-end via\nbackpropagation through time. The central (attractor) network employs Hebbian learning to rapidly\nremember the conjunction of location and sensory stimulus. A generative temporal model learns how\nto use the Hebbian memory most ef\ufb01ciently given the common statistics of transitions across worlds.\n\n3.2 Generative model\n\n(cid:0)x\u2264T , p\u2264T , g\u2264T\n\n(cid:1) =(cid:81)T\n\n(cid:0)gt | gt\u22121, at\n\n(cid:1) where observed\n\nt=1 p\u03b8 (xt | pt) p\u03b8 (pt | Mt\u22121, gt) p\u03b8\n\nWe consider the agent to have a generative model (Fig 3a, schematic in Figs 2b, 2c) which factorises\nas p\u03b8\nvariable xt is the instantaneous sensory stimulus and latent variables gt and pt are grid and place\ncells. Mt represents the agent\u2019s memory composed from past place cell representations. at represents\nthe current action - our version of head-direction cells [35]. \u03b8 are parameters of the generative model.\nWe now give concise, but intuitive descriptions of the model components. Expanded details in\nSupplementary Materials (SM). Sensory data xt is a one-hot vector where each of its ns elements\nrepresent a sensory identity. We consider place and grid cells, pt and gt respectively, to come\nin different frequencies (hierarchies) indexed by superscript f. Though we already refer to these\nvariables as grid and place cells, it is important to note that grid-ness and place-ness are not hard-coded\n- all representations are learned. f (\u00b7\u00b7\u00b7 ) denotes functions speci\ufb01c to the distribution in question.\n\n4\n\nMEC (gt)HPC (pt)LEC (xt)Mt?t \u2192 t+1\fGrid cells. To predict where we will be, we can transition from our current location based\non our heading (i.e. path integration, schematic in Fig 2c). p\u03b8\ntransition probability density, with transitions taking the form gt = f\u00b5g (gt\u22121 +Da gt\u22121) +\n\u03c3 \u00b7 \u03b5t with \u03b5t \u223c N (0, I), V ec[Da] = fD(at) and \u03c3 = f\u03c3g (gt\u22121). Connections in Da\nare from low frequency to the same or higher frequency only (or alternatively only within\nfrequency). We separate into hierarchical scales so that high frequency statistics can be\nlearning and knowledge is reused across space.\nreused across lower frequency statistics, i.e.\n\n(cid:0)gt | gt\u22121, at\n\n(cid:1) is a Gaussian\n\nPlace cells. p\u03b8 (pt | Mt\u22121, gt) is a Gaussian\nprobability density for retrieving memories.\nStored memories are extracted via an attractor\nnetwork (Fig 2b) using fg(gt) as input - i.e.\ngrid cells act as an index for memory extrac-\ntion (Details in Section 3.4).\nData. We classify a stimulus identity using\np\u03b8 (xt | pt) \u223c Cat (fx(pt)).\n\n3.3 Inference network\n\n(a) Generative model\n\n(b) Inference network\n\n(cid:0)gt | x\u2264t, Mt\u22121, gt\u22121\n\nt=1 q\u03c6\n\n(cid:0)g\u2264T , p\u2264T | x\u2264T\n\n(cid:1) = (cid:81)T\n\n(cid:1) q\u03c6 (pt | x\u2264t, gt). See Fig 3b for infer-\n\nFigure 3: Circled/ boxed variables are stochastic/ de-\nterministic. Red arrows indicate additional inference\ndependencies. Dashed arrows continue through time.\nDotted arrow are optional as explained below.\n\nDue to the inclusion of memories, as\nwell as other non-linearities, the posterior\np (gt, pt | x\u2264t, a\u2264t) is intractable - we there-\nfore turn to approximate inference. To infer\non this generative model, we make critical de-\ncisions that respect our proposal of structural information separated from sensory information as\nwell as respecting biological considerations. We use a recognition distribution that factorises as\nq\u03c6\nence network schematic. \u03c6 denote parameters of the inference network. We learn \u03b8 and \u03c6, by\nmaximising the ELBO with the variational autoencoder framework [20, 30] (details in SM).\nPlace cells. We treat these variables as a conjunction between sensorium and structural information\nfrom grid cells (Fig 2a). The sensorium is obtained by \ufb01rst compressing the immediate sensory data,\nxt, to fc(xt), after which it is \ufb01ltered via exponential smoothing into different frequency bands,\nxf\nt . After a normalisation step, each xf\nt to give the mean of the\ndistribution q\u03c6 (pt | x\u2264t, gt). The separation into hierarchical scales helps to provide a unique code\nfor each position, even if the same stimulus appears in several locations of one environment, since the\nsurrounding stimuli, and therefore the lower frequency place cells, are likely to be different. Since\nthe place cell representations form memories, one can utilise the hierarchical scales for memory\nretrieval. We note that although the exponential smoothing appears over-simpli\ufb01ed, it approximates\nthe Laplace transform with real coef\ufb01cients. Cells of this nature have been discovered in LEC [37].\nGrid cells. We factorise q\u03c6\nknow where we are, we can path integrate (q\u03c6\ntribution described above) as well as use sensory information that we may have seen previously\n(q\u03c6 (gt | x\u2264t, Mt\u22121)). The second distribution (optional addition) provides information on location\ngiven the sensorium. Since memories link location and sensorium, successfully retrieving a memory\ngiven sensory input allows us to re\ufb01ne our location estimate. Experimentally this improves training.\n\n(cid:1) as q\u03c6\n(cid:0)gt | gt\u22121, at\n\n(cid:0)gt | gt\u22121, at\n(cid:1) q\u03c6 (gt | x\u2264t, Mt\u22121). To\n(cid:1) - equivalent to the generative dis-\n\n(cid:0)gt | x\u2264t, Mt\u22121, gt\u22121\n\nt is combined conjunctively with gf\n\n3.4 Hebbian memories\n\nStorage. Memories of place cell representations are stored in Hebbian weights between place cells\n(Mt in Fig 2a), similar to [3]. We choose Hebbian learning, not only for its biological plausibility,\nbut to also allow rapid learning when entering a new environment . We use the following learning\nrule to update the memory: Mt = \u03bb Mt\u22121 +\u03b7(pt \u2212\u02c6pt)(pt +\u02c6pt)T , where \u02c6pt represents place cells\ngenerated from inferred grid cells. \u03bb and \u03b7 are the rate of forgetting and remembering respectively.\nConnections from high to low frequencies are set to zero, so that memories are retrieved hierarchically.\nWe note than many other types of Hebbian rules work. In SM we describe changes to the learning\nrule if the additional distribution q\u03c6 (gt | x\u2264t, Mt\u22121) is included for inference of grid cells.\n\n5\n\nMt-1Mtgtgt+1ptpt+1xtxt+1atat+1Mt-1Mtgtgt+1ptpt+1xtxt+1xft+1xftatat+1\f(a) Schematic\n\n(b) Grid cells\n\n(c) Banding\n\nFigure 4: Top panel all one environment, bottom panels another environment. (a) Schematic of two\nenvironments (not actual size). (b) Same three grid cells from each environment. High frequency grid\non left, lower frequency on the right. Same grid code used in environments of different sizes implies\na general way of representing space, i.e. not just a template of each environment. These are square\ngrids as we have chosen a four way embedding of actions and four connected space. (c) Banded cell.\n\nRetrieval. To retrieve memories, similarly to [3], we use an attractor network of the form h\u03c4 =\nfp (\u03b1h\u03c4\u22121 + Mt h\u03c4\u22121), where \u03c4 is the iteration of the attractor network and \u03b1 is a decay term.\nThe input to the attractor, h0, is from the grid cells or sensorium (with their dimensions scaled\nappropriately) depending on whether memories are being retrieved for generative or inference\npurposes respectively. The output of the attractor is the retrieved memory (place cell code).\n\n3.5 Model implications\n\nWe offer a solution to the problem of how structural codes are shared, via grid cells, to remapped\nplace cells. Even with identical structure, since sensory stimuli across environments are different, the\nconjunctive code is different. Thus we believe that place cell remapping is not random, instead place\ncells are chosen that are consistent with both grid and sensory codes. This is a different notion to\nother remapping models [1], where random grid modular realignment produces a new set of place\ncells, and learning, that anchors these new representations, starts afresh in each environment. Our\nmethod allows for dramatically faster learning, as learnt structure can be re-used in new environments.\nIn section 5, we present experimental evidence in concordance with our model.\nIn addition to offering a novel theory for place cell remapping, our model also provides an explanation\nfor what determines place \ufb01eld sizes. Speci\ufb01cally, a given place cell will be active in the regions\nof space that are consistent with both with the grid representation (structure) received by that place\ncell and the sensory experience coded for by that place cell. It further offers explanation for why a\ngiven place cell may have multiple place \ufb01elds within one environment, as there may be multiple\nlocations where this consistency holds. Therefore our model offers a novel framework for designing\nexperiments to manipulate place \ufb01eld sizes and locations, for example, based on simultaneously\nrecorded grid cells and environmental cues.\nWe believe that using more biologically realistic computational mechanisms (e.g. Hebbian Memory,\nno LSTM) will facilitate further incorporation of neuroscience-inspired phenomena, such as successor\nrepresentations or replay, which may be useful for building AI systems.\n\n4 Model experiments\n\nWe show that by predicting sensory observations in environments that share structure, the model\nlearns to generalise structural knowledge. This knowledge is represented similarly to spatial cells\nobserved in the brain, suggesting these cells play a key role in generalisation. We further show our\nmodel exhibits fast (one-shot) learning and performs inference of unseen relationships. Although we\npresented a probabilistic formulation, best results were obtained when only considering the means of\neach distribution. Further implementation details in SM. We have taken a didactic approach to our\nmodel, thus we do not expect stellar model performance, nevertheless the model performs well.\n\n6\n\n\f(b) Landmark cells\n\n(c) Object vector cells\n\n(d) Border cells\n\n(a) Place cell remapping\n\nFigure 5: Hippocampal cells (a, b) depend on sensory experience, whereas entorhinal cells (c, d)\ngeneralise over sensory experience. Example cells from two different environments (top/bottom).\na) Place cells demonstrating remapping, as also observed in the brain. These are typical in the\nmodel. Left/right: High/low frequency cell. b) Hippocampal landmark cells \ufb01re at a speci\ufb01c distance\nand direction from objects, not generalising over object identities. c) Object vector cells, however,\ngeneralise both within and across environments. d) Border cells.\n\nLearned spatial representations. We show the representations learned by our network in Fig 4\nand 5 by plotting spatial activity maps of particular neurons. In Fig 4b we present grid cells. The\ntop panel shows cells from one environment, and the bottom panels from a different and slightly\nsmaller environment. We see that our network chooses to represent information in a grid-like pattern\n(square-grids as the statistics of our space is square). We can also observe spatial \ufb01ring \ufb01elds at\ndifferent frequencies. Representations are consistent across environments, regardless of their size\n- thus we have a generalisable representation of 2D space, not just a template of a particular sized\nenvironment. Fig 4c shows banded cells from our model which are also observed in the brain\nalongside grid cells [24]. Further learned representations are shown in SM.\nWe observe the appearance of phases in the grid cells (middle and right panels of Fig 4b), i.e. we \ufb01nd\ngrid representations that are shifted versions of each other, as in the brain [16]. The separation into\ndifferent phases means that two conjunctive place cells that respond to the same stimulus, will not\nnecessarily be active simultaneously - each cell will only be active when their corresponding grid\nphase is active. Thus one can uniquely code for the same stimulus in many different locations. Across\ntwo environments, a given stimulus may occur at the same grid phase but at a different location. Thus,\ndue to their conjunctive nature, place cells may remap across environments, as in the brain. We show\nthis in Fig 5a. Further learned place representations are shown in SM.\nTransition statistics determine basis functions. By changing transition statistics, other entorhinal\ncell types are observed in our model. Encouraging the agent to spend more time near boundaries leads\nto the emergence of border cells [31] (Fig 5d). Biasing towards particular sensory experiences leads\nto the discovery of object vector cells [19] (Fig 5c). Similarly to experimental evidence, although\nthese object vector cells in our \u2019grid\u2019 cell layer generalise over objects, the equivalent landmark cells\n[11] in our \u2019place\u2019 cell layer do not - they are object speci\ufb01c (Fig 5b). Our results suggest that the\n\u2019zoo\u2019 of different cell types found in entorhinal cortex may be viewed under a uni\ufb01ed framework\n- summarising the common statistics of tasks into basis functions that can be \ufb02exibly combined\ndepending on the particular structural constraints of the environment the animal/agent faces. After\nan initial guess of task structure, appropriate weighting of the bases can be inferred on-line (e.g. by\nsensory cues / performance) to parsimoniously describe the current task structure.\nOne-shot learning. We test whether the network can remember what it has just seen. We consider\noccasions when the agent stays still at a node for the \ufb01rst time, as a function of the number of times\nthat node has previously been visited (Fig 6a). We see that the agent is able to predict at a high\naccuracy even if it has only just visited the node for the \ufb01rst time. This indicates we are able to do\none-shot-learning with Hebbian memory, demonstrating our model can learn episodic memories.\nZero-shot inference. Having learned the structure of our space, we should be able to correctly\npredict previously visited nodes even if we approach from a non-traversed edge - i.e. infer a link\nin the graph on loop closure. We present such data in Fig 6b. We plot the prediction accuracy of\nsuch link inferences as a function of the fraction of the total nodes visited in the graph. We achieve\nconsiderably better than chance (1/ns = 0.02) prediction, which remains stable throughout graph\ntraversal. This shows that structural information is used for inferring unseen relationships.\nLong term memories. Despite using BPTT truncated at 25 steps, we retain memories for much\nlonger (Fig 6c), indicating our grid code allows ef\ufb01cient storage and retrieval of episodic memories.\n\n7\n\nEnv 1Env 2\f(a) One-shot learning\n\n(b) Zero-shot link inference\n\n(c) Long term memories\n\nFigure 6: Prediction accuracy for different box widths. (a), (b) are previously unseen links only. Black\ndashed line is chance. (a) Attractor network is immediately able to retrieve Hebbian memories. (b)\nUnobserved graph links are inferred, implying the network has successfully learned and generalised\nstructural knowledge. (c) Memories are successfully retrieved a long time after initial storage.\n\nTo reiterate, no representations are hard-coded. Place-like representations are learned in the attractor.\nGrid-like (and other entorhinal) representations are learned in the generative temporal model. These\nemerge from end-to-end training. These grid-like representations allow zero shot inference in new\nworlds demonstrating structural generalisation.\n\n5 Analysis of data from a remapping experiment\n\nOur framework predicts place cells and grid cells retain their relationship across environments,\nallowing generalisation of structure encoded by grid cells. We empirically test this prediction in\ndata from a remapping experiment [5] where both place and grid cells were recorded from rats\nin two different environments. The environments were of the same dimensions (1m by 1m) but\ndiffered in their sensory (texture/visual/olfactory) cues so the animals could distinguish between\nthem. Each of seven rats has recordings from both environments. Recordings on each day consist\nof \ufb01ve twenty-minute trials in the environments: the \ufb01rst and last trials in one environment and the\nintervening three trials in a second environment.\n\n5.1 Methods\n\nWe test the prediction that a given place cell retains its relationship with a given grid cell across\nenvironments using two measures. First, whether grid cell activity at the position of peak place cell\nactivity is correlated across environments (gridAtPlace), and second, whether the minimum distance\nbetween the peak place cell activity and a peak of grid cell activity is correlated across environments\n(minDist; normalised to corresponding grid scale). To account for potential confounds or biases (e.g.\nborder effects, inaccurate peaks), we \ufb01t the recorded grid cell rate maps to an idealised grid cell\nequation [33], and use this ideal grid rate map to give grid cell \ufb01ring rates and locations of grid peaks.\nOnly grid cells with high grid scores (> 0.8) were used to ensure good ideal grid \ufb01ts to the data, and\nwe excluded grid cells with large scales (> 50cm), both computed as in [5]. Locations of place cell\npeaks were simply de\ufb01ned as the location of maximum activity in a given cell\u2019s rate map. To account\nfor border effects, we removed place cells that had peaks close to borders (< 10cm from a border).\nOur framework predicts a preserved relationship between place and grid cells of the same spatial\nscale (module). However, since we do not know the modules of the recorded cells, we can only\nexpect a non-random relationship across the entire population. For each measure, we compute its\nvalue for every place cell-grid cell pair (from two trials). A correlation across trials is then performed\non these values. To test the signi\ufb01cance of this correlation and ensure it is not driven by bias in the\ndata, we generate a null distribution by randomly shifting the place cell rate maps and recomputing\nthe measures and their correlation across trials. We then examine where the correlation of the\nnon-shuf\ufb02ed data lies relative to the null.\n\n5.2 Results\n\nWe present analyses for both the gridAtPlace measure (Fig 7a) and the minDist measure (Fig 7b).\nThe scatter plots show the correlation of a given measure across trials, where each point is a place\ncell-grid cell pair. The histogram plots show where this correlation (green line) lies relative to the\nnull distribution of correlation coef\ufb01cients. The p value is the proportion of the null distribution that\nis greater than the unshuf\ufb02ed correlation.\n\n8\n\n1234# times node visited0.00.20.40.60.81.0Prediction accuracy when staying still810120.00.20.40.60.81.0Proportion of nodes visited0.00.20.40.60.81.0Correct inference of link81012010204060100200400# steps since visited0.00.20.40.60.81.0Prediction accuracy81012\f(a) Grid at place\n\n(c) Model-data correspondence\nFigure 7: (a), (b) Data analysis results: top panels are for within environment analyses, bottom panels\nacross environment analyses. (c) Black/ Red: Data/ Model. Top: gridAtPlace across environments.\nBottom: Scatter of elements of correlation matrices across environments.\n\n(b) Minimum distance\n\nAs a sanity check, we \ufb01rst con\ufb01rm these measures are signi\ufb01cantly correlated within environments\n(i.e. across two visits to the same environment - trials 1 and 5), when the cell populations should not\nhave remapped (see Fig 7a, top and 7b, top). We then test across environments (i.e. two different\nenvironments - trials 1 and 4), to asses whether our predicted non-random remapping relationship\nbetween grid and place cells exists (Fig 7a, bottom and 7b, bottom). Here we also \ufb01nd signi\ufb01cant\ncorrelations for both measures for the 115 place cell-grid cell pairs. We note the gridAtPlace result\nholds across environments (p < 0.005) when not \ufb01tting an ideal grid and using a wide range of grid\nscore cut-offs (minDist not calculated without the ideal grid due to inaccurate grid peaks). Finally\nperforming the across environment gridAtPlace analysis with our model rate maps (Fig 7c top), we\nobserve correlations of 0.3-0.35, which are consistent with that of the data.\nTo share structure, the relationship between grid cells should be preserved across environments. The\ngrid cell correlation matrix is preserved (i.e. itself correlated) across environments (p < 0.001 from\nnull), both in the data [5] as well as in our model (Fig 7c bottom). These results are consistent with\nthe notion that grid cells encode generalisable structural knowledge.\nThese are the \ufb01rst analyses demonstrating non-random place cell remapping based on neural activity,\nand provide evidence for a key prediction of our model: that place cells, despite remapping across\nenvironments, retain their relationship with grid cells.\n\n6 Conclusions\n\nWe proposed a mechanism for generalisation of structure inspired by the hippocampal-entorhinal\nsystem. We proposed that one can generalise state-space statistics via explicit separation of structure\nand stimuli, while using a conjunctive representation with fast memory to link the two. We proposed\nthat spatial hierarchies are utilised to allow for an ef\ufb01cient combinatorial code. We have shown\nthat hierarchical grid-like and place-like representations emerge naturally from our model in a\npurely unsupervised learning setting. We have shown that these representations are effective at both\ngeneralising the state-space (zero-shot link inference), but also for hierarchical memory addressing.\nWe have proposed that entorhinal cortex provides a basis set for describing the current transition\nstructure, unifying many entorhinal cell types. We have suggested that spatial coding is just one\ninstance of a broader framework organising knowledge. Our framework incorporates numerous\nphenomena or functions ascribed to the hippocampal formation (spatial cognition and representations,\nconceptual knowledge representation, hierarchical representations, episodic memory, inference, and\ngeneralisation). We have also presented experimental evidence that demonstrates grid and place\ncells retain their relationships across environment, which supports our model assumptions. We hope\nthat this work can provide new insights that will allow for advances in AI, as well as providing new\npredictions, constraints and understanding in neuroscience.\n\n7 Author Contributions\n\nJCRW developed the model, performed simulations and drafted paper. CB collected data. JCRW,\nTHM analysed data. JCRW, THM, SM, TEJB conceived project and edited paper. TEJB supervised.\n\n9\n\n00.51gridAtPlace in env 1 (a.u.)00.20.40.60.81gridAtPlace in env 1' (a.u.)Correlation = 0.38618-0.4-0.200.20.4gridAtPlace correlation coefficients051015202530numberp-value : 000.51gridAtPlace in env 1 (a.u.)00.20.40.60.81gridAtPlace in env 2 (a.u.)Correlation = 0.32046-0.4-0.200.20.4gridAtPlace correlation coefficients051015202530numberp-value : 0.00200.51gridAtPlace in env 1 (a.u.)00.20.40.60.81gridAtPlace in env 1' (a.u.)Correlation = 0.38618-0.4-0.200.20.4gridAtPlace correlation coefficients051015202530numberp-value : 000.51gridAtPlace in env 1 (a.u.)00.20.40.60.81gridAtPlace in env 2 (a.u.)Correlation = 0.32046-0.4-0.200.20.4gridAtPlace correlation coefficients051015202530numberp-value : 0.00200.0050.01minDist in env 1 (a.u.)00.0020.0040.0060.0080.010.012minDist in env 1' (a.u.)Correlation = 0.37741-0.4-0.200.20.4minDist correlation coefficients010203040numberp-value : 000.0050.01minDist in env 1 (a.u.)00.0020.0040.0060.0080.010.012minDist in env 2 (a.u.)Correlation = 0.16974-0.4-0.200.20.4minDist correlation coefficients010203040numberp-value : 0.02800.0050.01minDist in env 1 (a.u.)00.0020.0040.0060.0080.010.012minDist in env 1' (a.u.)Correlation = 0.37741-0.4-0.200.20.4minDist correlation coefficients010203040numberp-value : 000.0050.01minDist in env 1 (a.u.)00.0020.0040.0060.0080.010.012minDist in env 2 (a.u.)Correlation = 0.16974-0.4-0.200.20.4minDist correlation coefficients010203040numberp-value : 0.028\f8 Acknowledgements\n\nWe acknowledge funding from a Wellcome Trust Senior Research Fellowship (WT104765MA)\ntogether with a James S. McDonnell Foundation Award (JSMF220020372) to TEJB, MRC scholarship\nto THM, and an EPSRC scholarship to JCRW.\n\nReferences\n[1] Larry F. Abbott, J. D. Monaco, and Larry F. Abbott. Modular Realignment of Entorhinal Grid Cell\nActivity as a Basis for Hippocampal Remapping. Journal of Neuroscience, 31(25):9414\u20139425, 2011. ISSN\n0270-6474. doi: 10.1523/JNEUROSCI.1433-11.2011. URL http://www.jneurosci.org/cgi/doi/\n10.1523/JNEUROSCI.1433-11.2011.\n\n[2] Dmitriy Aronov, Rhino Nevers, and David W. Tank. Mapping of a non-spatial dimension by the hippocam-\npal\u2013entorhinal circuit. Nature, 543(7647):719\u2013722, 2017. ISSN 0028-0836. doi: 10.1038/nature21692.\nURL http://www.nature.com/doifinder/10.1038/nature21692.\n\n[3] Jimmy Lei Ba, Geoffrey Hinton, Volodymyr Mnih, Joel Z. Leibo, and Catalin Ionescu. Using Fast Weights\nto Attend to the Recent Past. Advances in Neural Information Processing Systems, pages 1\u201310, 2016. URL\nhttp://arxiv.org/abs/1610.06258.\n\n[4] Andrea Banino, Caswell Barry, Benigno Uria, Charles Blundell, Timothy Lillicrap, Piotr Mirowski,\nAlexander Pritzel, Martin J Chadwick, Thomas Degris, Joseph Modayil, Greg Wayne, Hubert Soyer, Fabio\nViola, Brian Zhang, Ross Goroshin, Neil Rabinowitz, Razvan Pascanu, Charlie Beattie, Stig Petersen, Amir\nSadik, Stephen Gaffney, Helen King, Koray Kavukcuoglu, Demis Hassabis, Raia Hadsell, and Dharshan\nKumaran. Vector-based navigation using grid-like representations in arti\ufb01cial agents. Nature, 557(7705):\n429\u2013433, may 2018. ISSN 0028-0836. doi: 10.1038/s41586-018-0102-6. URL http://dx.doi.org/\n10.1038/s41586-018-0102-6http://www.nature.com/articles/s41586-018-0102-6.\n\n[5] C. Barry, L. L. Ginzberg, J. O\u2019Keefe, and N. Burgess. Grid cell \ufb01ring patterns signal environmental novelty\nby expansion. Proceedings of the National Academy of Sciences, 109(43):17687\u201317692, 2012. ISSN\n0027-8424. doi: 10.1073/pnas.1209918109. URL http://www.pnas.org/cgi/doi/10.1073/pnas.\n1209918109.\n\n[6] E Bostock, R U Muller, and J L Kubie. Experience-dependent modi\ufb01cations of hippocampal place cell\n\n\ufb01ring. Hippocampus, 1(2):193\u2013205, 1991. ISSN 1050-9631. doi: 10.1002/hipo.450010207.\n\n[7] Cindy A. Buckmaster, Howard Eichenbaum, David G Amaral, Wendy A Suzuki, and Peter R Rapp.\nEntorhinal Cortex Lesions Disrupt the Relational Organization of Memory in Monkeys. Journal of\nNeuroscience, 24(44):9811\u20139825, 2004. ISSN 0270-6474. doi: 10.1523/JNEUROSCI.1532-04.2004. URL\nhttp://www.jneurosci.org/cgi/doi/10.1523/JNEUROSCI.1532-04.2004.\n\n[8] Daniel Bush, Caswell Barry, Daniel Manson, and Neil Burgess. Using Grid Cells for Navigation. Neuron,\n87(3):507\u2013520, 2015. ISSN 10974199. doi: 10.1016/j.neuron.2015.07.006. URL http://dx.doi.org/\n10.1016/j.neuron.2015.07.006.\n\n[9] Alexandra O. Constantinescu, Jill X. O\u2019Reilly, and Timothy E. J. Behrens. Organizing conceptual\nknowledge in humans with a gridlike code. Science, 352(6292):1464\u20131468, 2016. ISSN 10959203. doi:\n10.1126/science.aaf0941.\n\n[10] Christopher J. Cueva and Xue-Xin Wei. Emergence of grid-like representations by training recurrent neural\nnetworks to perform spatial localization. pages 1\u201319, 2018. URL http://arxiv.org/abs/1803.07770.\n\n[11] Sachin S Deshmukh and James J Knierim.\n\nIn\ufb02uence of local objects on hippocampal representa-\ntions: Landmark vectors and memory. Hippocampus, 23(4):253\u201367, apr 2013.\nISSN 1098-1063.\ndoi: 10.1002/hipo.22101. URL http://www.ncbi.nlm.nih.gov/pubmed/23447419http://www.\npubmedcentral.nih.gov/articlerender.fcgi?artid=PMC3869706.\n\n[12] Yedidyah Dordek, Daniel Soudry, Ron Meir, and Dori Derdikman. Extracting grid cell characteristics from\nplace cell inputs using non-negative principal component analysis. eLife, 5(MARCH2016):1\u201336, 2016.\nISSN 2050084X. doi: 10.7554/eLife.10094.\n\n[13] J. A. Dusek and H. Eichenbaum. The hippocampus and memory for orderly stimulus relations. Proceedings\nof the National Academy of Sciences, 94(13):7109\u20137114, 1997. ISSN 0027-8424. doi: 10.1073/pnas.94.13.\n7109. URL http://www.pnas.org/cgi/doi/10.1073/pnas.94.13.7109.\n\n10\n\n\f[14] Marianne Fyhn, Torkel Hafting, Alessandro Treves, May Britt Moser, and Edvard I. Moser. Hippocampal\nremapping and grid realignment in entorhinal cortex. Nature, 446(7132):190\u2013194, 2007. ISSN 14764687.\ndoi: 10.1038/nature05601.\n\n[15] Mevlana Gemici, Chia-Chun Hung, Adam Santoro, Greg Wayne, Shakir Mohamed, Danilo J. Rezende,\nDavid Amos, and Timothy Lillicrap. Generative Temporal Models with Memory. pages 1\u201325, 2017. ISSN\n1702.04649. URL http://arxiv.org/abs/1702.04649.\n\n[16] Torkel Hafting, Marianne Fyhn, Sturla Molden, May-britt Britt Moser, and Edvard I. Moser. Microstructure\nof a spatial map in the entorhinal cortex. Nature, 436(7052):801\u2013806, 2005. ISSN 00280836. doi:\n10.1038/nature03721.\n\n[17] Demis Hassabis, Dharshan Kumaran, S. D. Vann, and E. A. Maguire. Patients with hippocampal amnesia\ncannot imagine new experiences. Proceedings of the National Academy of Sciences, 104(5):1726\u20131731,\n2007. ISSN 0027-8424. doi: 10.1073/pnas.0610561104. URL http://www.pnas.org/cgi/doi/10.\n1073/pnas.0610561104.\n\n[18] Demis Hassabis, Dharshan Kumaran, Christopher Summer\ufb01eld, and Matthew Botvinick. Neuroscience-\nInspired Arti\ufb01cial Intelligence. Neuron, 95(2):245\u2013258, 2017. ISSN 10974199. doi: 10.1016/j.neuron.\n2017.06.011. URL http://dx.doi.org/10.1016/j.neuron.2017.06.011.\n\n[19] \u00d8yvind Arne H\u00f8ydal, Emilie Ranheim Skyt\u00f8en, M B Moser, and Edvard I Moser. Object-vector cells in\n\nthe medial entorhinal cortex. bioRxiv, 2018. doi: 10.1101/286286.\n\n[20] Diederik P. Kingma and Max Welling. Auto-Encoding Variational Bayes.\n\n1312.6114v10. doi: 10.1051/0004-6361/201527329. URL http://arxiv.org/abs/1312.6114.\n\n(Ml):1\u201314, 2013.\n\nISSN\n\n[21] Kirsten Brun Kjelstrup, Trygve Solstad, Vegard Heimly Brun, Torkel Hafting, Stefan Leutgeb, Menno P.\nWitter, Edvard I. Moser, and May-Britt Moser. Finite Scale of Spatial Represenation in the Hippocampus.\nScience, 321(July):140 \u2013 143, 2008. ISSN 0036-8075.\n\n[22] Robert W. Komorowski, Joseph R. Manns, and Howard Eichenbaum. Robust Conjunctive Item-Place\nCoding by Hippocampal Neurons Parallels Learning What Happens Where. Journal of Neuroscience,\n29(31):9918\u20139929, 2009. ISSN 0270-6474. doi: 10.1523/JNEUROSCI.1378-09.2009. URL http:\n//www.jneurosci.org/cgi/doi/10.1523/JNEUROSCI.1378-09.2009.\n\n[23] Alex Krizhevsky, Ilya Sutskever, and Geoffrey E Hinton. ImageNet Classi\ufb01cation with Deep Convolutional\nNeural Networks. Advances In Neural Information Processing Systems, pages 1\u20139, 2012. ISSN 10495258.\ndoi: http://dx.doi.org/10.1016/j.protcy.2014.09.007.\n\n[24] Julia Krupic, Neil Burgess, and John O\u2019Keefe. Neural Representations of Location Composed of Spatially\nPeriodic Bands. 337(August):853\u2013857, 2012. URL http://www.sciencemag.org/content/337/\n6096/853.full.pdf.\n\n[25] Dharshan Kumaran, Jennifer J. Summer\ufb01eld, Demis Hassabis, and Eleanor A. Maguire. Tracking the\nEmergence of Conceptual Knowledge during Human Decision Making. Neuron, 63(6):889\u2013901, 2009.\nISSN 08966273. doi: 10.1016/j.neuron.2009.07.030. URL http://dx.doi.org/10.1016/j.neuron.\n2009.07.030.\n\n[26] Stefan Leutgeb, Jill K. Leutgeb, Carol A. Barnes, Edvard I. Moser, Bruce L. McNaughton, and May-Britt\nMoser. Independent Codes for Spatial and Episodic Memory in Hippocampal Neuronal Ensembles. (July):\n619\u2013624, jul 2005.\n\n[27] James L. McClelland, Bruce L. McNaughton, and Randall C. O\u2019Reilly. Why there are complementary\nlearning systems in the hippocampus and neocortex: Insights from the successes and failures of connec-\ntionist models of learning and memory. Psychological Review, 102(3):419\u2013457, 1995. ISSN 0033295X.\ndoi: 10.1037/0033-295X.102.3.419.\n\n[28] Volodymyr Mnih, Koray Kavukcuoglu, David Silver, Andrei A Rusu, Joel Veness, Bellemare Marc G,\nAlex Graves, Martin Riedmiller, Andreas K. Fidjeland, Georg Ostrovski, Stig Petersen, Charles Beattie,\nAmir Sadik, Ioannis Antonoglou, Helen King, Dharshan Kumaran, Daan Wierstra, Shane Legg, and Demis\nHassabis. Human-level control through deep reinforcement learning. Nature, 518(7540):529\u2013533, 2015.\nISSN 10450823. doi: 10.1038/nature14236. URL http://dx.doi.org/10.1038/nature14236.\n\n[29] John O\u2019Keefe and J. Dostrovsky. The hippocampus as a spatial map. Preliminary evidence from\nunit activity in the freely-moving rat. Brain Research, 34(1):171\u2013175, nov 1971.\nISSN 00068993.\ndoi: 10.1016/0006-8993(71)90358-1. URL http://linkinghub.elsevier.com/retrieve/pii/\n0006899371903581.\n\n11\n\n\f[30] Danilo Jimenez Rezende, Shakir Mohamed, and Daan Wierstra. Stochastic Backpropagation and Approxi-\nmate Inference in Deep Generative Models. 2014. ISSN 10495258. doi: 10.1051/0004-6361/201527329.\nURL http://arxiv.org/abs/1401.4082.\n\n[31] Trygve Solstad, Charlotte N Boccara, Emilio Kropff, May-Britt Moser, and Edvard I Moser. Representation\nof geometric borders in the entorhinal cortex. Science (New York, N.Y.), 322(5909):1865\u20138, dec 2008. ISSN\n1095-9203. doi: 10.1126/science.1166466. URL papers3://publication/doi/10.1126/science.\n1166466http://www.ncbi.nlm.nih.gov/pubmed/19095945.\n\n[32] Kimberley L. Kimberly L. Stachenfeld, Matthew M. Botvinick, and Samuel J. Gershman. The hippocampus\nas a predictive map. Nature Neuroscience, 20(11):1643\u20131653, 2017. ISSN 15461726. doi: 10.1038/nn.\n4650.\n\n[33] Martin Stemmler, Alexander Mathis, and Andreas Herz. Connecting Multiple Spatial Scales to Decode the\nPopulation Activity of Grid Cells. Science Advances, in press(December):1\u201312, 2015. ISSN 2375-2548.\ndoi: 10.1126/science.1500816.\n\n[34] Hanne Stensola, Tor Stensola, Trygve Solstad, Kristian Fr\u00d8land, May-Britt Britt Moser, and Edvard I.\nMoser. The entorhinal grid map is discretized. Nature, 492(7427):72\u201378, 2012.\nISSN 00280836.\ndoi: 10.1038/nature11649. URL http://www.nature.com/doifinder/10.1038/nature11649http:\n//dx.doi.org/10.1038/nature11649.\n\n[35] J S Taube, Robert U Muller, and J B Ranck. Head-direction cells recorded from the postsubiculum in freely\nmoving rats. I. Description and quantitative analysis. The Journal of neuroscience : the of\ufb01cial journal of\nthe Society for Neuroscience, 10(2):420\u201335, 1990. ISSN 0270-6474. doi: 10.1212/01.wnl.0000299117.\n48935.2e. URL http://www.ncbi.nlm.nih.gov/pubmed/2303851.\n\n[36] Edward C. Tolman. Cognitive maps in rats and men. Psychological Review, 55(4):189\u2013208, 1948.\nISSN 1939-1471. doi: 10.1037/h0061626. URL http://doi.apa.org/getdoi.cfm?doi=10.1037/\nh0061626.\n\n[37] Albert Tsao, J\u00f8rgen Sugar, Li Lu, Cheng Wang, James J. Knierim, May-Britt Moser, and Edvard I. Moser.\nIntegrating time from experience in the lateral entorhinal cortex. Nature, 2018. ISSN 0028-0836. doi:\n10.1038/s41586-018-0459-6. URL http://www.nature.com/articles/s41586-018-0459-6.\n\n[38] James C. R. Whittington and Rafal Bogacz. An Approximation of the Error Backpropagation\nNeural\ndoi: 10.1162/NECO_a_00949.\nhttp://www.mitpressjournals.org/doi/10.1162/NECO{_}a{_}00949https://www.\n\nAlgorithm in a Predictive Coding Network with Local Hebbian Synaptic Plasticity.\nComputation, 29(5):1229\u20131262, may 2017.\nURL\nbiorxiv.org/content/early/2016/12/23/035451https://www.ncbi.nlm.nih.gov/pubmed/\n28333583{%}0Ahttps://www.ncbi.nlm.nih.gov/pmc/articles/PMC5467749/pdf/emss-73.\n\nISSN 0899-7667.\n\n[39] Emma R. Wood, Paul A. Dudchenko, and Howard Eichenbaum. The global record of memory in\nhippocampal neuronal activity. Nature, 397(6720):613\u2013616, 1999. ISSN 00280836. doi: 10.1038/17605.\n\n12\n\n\f", "award": [], "sourceid": 5133, "authors": [{"given_name": "James", "family_name": "Whittington", "institution": "University of Oxford"}, {"given_name": "Timothy", "family_name": "Muller", "institution": "University of Oxford"}, {"given_name": "Shirely", "family_name": "Mark", "institution": "University College London"}, {"given_name": "Caswell", "family_name": "Barry", "institution": "University College London"}, {"given_name": "Tim", "family_name": "Behrens", "institution": "University of Oxford"}]}