{"title": "Norepinephrine and Neural Interrupts", "book": "Advances in Neural Information Processing Systems", "page_first": 243, "page_last": 250, "abstract": "", "full_text": "Norepinephrine and Neural Interrupts\n\nPeter Dayan\n\nGatsby Computational Neuroscience Unit\n\nUniversity College London\n\n17 Queen Square, London WC1N 3AR, UK\n\ndayan@gatsby.ucl.ac.uk\n\nAbstract\n\nAngela J. Yu\n\nCenter for Brain, Mind & Behavior\nGreen Hall, Princeton University\n\nPrinceton, NJ 08540, USA\najyu@princeton.edu\n\nExperimental data indicate that norepinephrine is critically involved in\naspects of vigilance and attention. Previously, we considered the func-\ntion of this neuromodulatory system on a time scale of minutes and\nlonger, and suggested that it signals global uncertainty arising from\ngross changes in environmental contingencies. However, norepinephrine\nis also known to be activated phasically by familiar stimuli in well-\nlearned tasks. Here, we extend our uncertainty-based treatment of nore-\npinephrine to this phasic mode, proposing that it is involved in the de-\ntection and reaction to state uncertainty within a task. This role of nore-\npinephrine can be understood through the metaphor of neural interrupts.\n\n1\n\nIntroduction\n\nTheoretical approaches to understanding neuromodulatory systems are plagued by the lat-\nter\u2019s neural ubiquity, evolutionary longevity, and temporal promiscuity. Neuromodulators\nact in potentially different ways over many different time-scales [14]. There are various\ngeneral notions about their roles, such as regulating sleeping and waking [13] and chang-\ning the signal to noise ratios of cortical neurons [11]. However, these are slowly giving\nway to more speci\ufb01c computational ideas [20, 7, 10, 24, 25, 5], based on such notions as\noptimal gain scheduling, prediction error and uncertainty.\n\nIn this paper, we focus on the short term activity of norepinephrine (NE) neurons in the\nlocus coeruleus [18, 1, 2, 3, 16, 4]. These neurons project NE to subcortical structures and\nthroughout the entire cortex, with individual neurons having massive axonal arborizations\n[12]. Over medium and short time-scales, norepinephrine is implicated in various ways in\nattention, vigilance, and learning. Given the widespread distribution and effects of NE in\nkey cognitive tasks, it is very important to understand what it is in a task that drives the\nactivity of NE neurons, and thus what computational effects it may be exerting.\n\nFigure 1 illustrates some of the key data that has motivated theoretical treatments of NE.\nFigure 1A;B;C show more tonic responses operating around a time-scale of minutes. Fig-\nures 1D;E;F show the short-term effects that are our main focus here.\n\nBrie\ufb02y, Figures 1A;B show that when the rules of a task are reversed, NE in\ufb02uences the\nspeed of adaptation to the changed contingency (Figure 1A) and the activity of noradrener-\ngic cells is tonically elevated (Figure 1B). Based on these data, we suggested [24, 25] that\nmedium-term NE reports unexpected uncertainty arising from unpredicted changes in an\nenvironment or task. This signal is a key part of a strategy for inference in potentially labile\ncontexts. It operates in collaboration with a putatively cholinergic signal which reports on\nexpected uncertainty that arises, for instance, from known variability or noise.\n\n\fA\n\n100\n\n80\n\n60\n\n40\n\n20\n\nn\no\ni\nr\ne\nt\ni\nr\nc\ng\nn\nh\nc\na\ne\nr\n\ni\n\ns\nt\na\nr\n\n%\n\nD\n\nc\ne\ns\n/\ns\ne\nk\np\nS\n\ni\n\nIdazoxan\nSaline\n\n0\n1\n# days after spatial \u2192 visual shift\n\n10\n\n15\n\n5\n\ntarget\n\nnon-target\n\nC\n\nc\ne\ns\n/\ns\ne\nk\np\nS\n\ni\n\n0\n\n15\n\nTime (min)\n\n30\n\n)\nz\nH\n\n(\n\ne\nt\na\nr\n\nA\nF\n\nF\n\nTime (sec)\n\nB\n\nc\ne\ns\n/\ns\ne\nk\np\nS\n\ni\n\nE\n\nc\ne\ns\n/\ns\ne\nk\np\nS\n\ni\n\nTime\n\nTime (sec)\n\nTime\n\nFigure 1: NE activity and effects. (A) Rats solve a sequential decision problem in a linear\nmaze. When the relevant cues are switched after a few days of learning (from spatial to\nvisual), rats with pharmacologically boosted NE (\u201cidazoxan\u201d) learn to use the new set of\ncues faster than the controls. Adapted from [9]. (B) In a vigilance task, monkeys respond\nto rare targets and ignore common distractor stimuli. The trace shows the activity of a\nsingle NE neuron in the locus coeruleus (LC) around the time of a target-distractor rever-\nsal (vertical line). Tonic activity is elevated for a considerable period. Adapted from [2].\n(C) Correlation between the gross \ufb02uctuations in the tonic activity of a single NE neuron\n(upper) and performance in the task (lower, measured by false alarm rate). Adapted from\n[20]. (D) Single NE cells are activated on a phasic time-scale stimulus locked (vertical line)\nto the target (upper plot) and not the distractor (lower plot). Adapted from [16]. (E) The\naverage responses of a large number of norepinephrine cells (over a total of 41,454 trials)\nstimulus locked (vertical line) to targets or distractors, sorted by the nature and rectitude\nof the response. The asterisk marks (similar) early activation of the neurons by the stimu-\nlus. Adapted from [16]. (F) In a GO/NO-GO olfactory discrimination task for rats, single\nunits are activated by the target odor (and not by the distractor odor), but are temporally\nmuch more tightly locked to the response (right) than the stimulus (left). Trials are ordered\naccording to the time between stimulus (blue) and response (red). Adapted from [4].\n\nHowever, Figures 1D;E;F, along with other substantial neurophysiological data on the ac-\ntivity of NE neurons [18, 4], show NE neurons have phasic response properties that lie\noutside this model. The data in Figure 1D;E come from a vigilance task [1], in which\nsubjects can gain reward by reacting to a rare target (a rectangle oriented one way), while\nignoring distractors (a rectangle oriented in the orthogonal direction). Under these circum-\nstances, NE is consistently activated by the target and not the distractor (Figure 1D). There\nare also clear correlations in the magnitude of the NE activity and the nature of a trial: hit,\nmiss, false alarm, correct reject (Figure 1E). It is known that the activity is weaker if the tar-\ngets are more common [17] (though the lack of response to rare distractors shows that NE\nis not driven by mere rarity), and disappears if no action need be taken in response to the\ntarget [18]. In fact, the signal is more tightly related in time to the subsequent action than\nthe preceding stimulus (Figure 1F). The signal has been qualitatively described in terms of\nin\ufb02uencing or controlling the allocation of behavioral or cognitive resources [20, 4].\n\nSince it arises on every trial in an extremely well-learned task with stable stimulus contin-\ngencies, this NE signal clearly cannot be indicating unpredicted task changes. Brown et\n\n\fal [5] have recently made the seminal suggestion that it reports changes in the statistical\nstructure of the input (stimulus-present versus stimulus-absent) to decision-making circuits\nthat are involved in initiating differential responding to distinct target stimuli. A statisti-\ncally necessary consequence of the change in the input structure is that afferent information\nshould be integrated differently: sensory responses should be ignored if no target is present,\nbut taken seriously otherwise. Their suggestion is that NE, by changing the gain of neurons\nin the decision-making circuit, has exactly this optimizing effect.\n\nIn this paper, we argue for a related, but distinct, notion of phasic NE, suggesting that it\nreports on unexpected state changes within a task. This is a signi\ufb01cant, though natural,\nextension of its role in reporting unexpected task changes [25]. We demonstrate that it ac-\ncounts well for the neurophysiological data. In agreement with the various accounts of the\neffects of phasic NE, we consider its role as a form of internal interrupt signal [6]. Com-\nputers use interrupts to organize the correct handling of internal and external events such\nas timers or peripheral input. Higher-level programs specify what interrupts are allowed\nto gain control, and the consequences thereof. We argue that phasic NE is the medium for\na somewhat similar neural interrupt, allowing the correct handling of statistically atypical\nevents. This notion relates comfortably to many existing views of phasic NE, and provides\na computational correlate for quantitative models.\n\n2 The Model\n\nFigure 2A illustrates a simple hidden Markov generative model (HMM) of the vigilance\ntask in Figure 1B-E. The (start) state models the condition established when the mon-\nkey \ufb01xates the light and initiates a trial. Following a somewhat variable delay, either the\ntarget (target) or the distractor (distractor) is presented, and the monkey must respond\nappropriately (release a continuously depressed bar for target and continue pressing for\ndistractor) The transition out of start is uniformly distributed between timesteps 6 and 10,\nimplemented by a time-varying transition function for this node:\nst = start\nst = distractor\nst = target\n\nP (st|st\u22121 = start) = \uf8f1\uf8f2\n1 \u2212 qt\n0.8qt\n0.2qt\n\uf8f3\n\n(1)\n\nwhere qt = 1/(11\u2212t) for (6 \u2264 t \u2264 10) and qt = 0 otherwise. The start and target states are\nassumed to be absorbing states (self-transition probability = 1). This transition function\nensures that the stimulus onset has a uniform distribution between 6 and 10 timesteps (and\n0 otherwise). Given that a transition out of start (into either target or distractor) takes\nplace, the probability is .2 for entering target and .8 for start, as in the actual task.\nIn addition, it is assumed that the node start does not emit observations, while target emits\nxt = t with probability \u03b7 > 0.5 and d with probability 1 \u2212 \u03b7, and distractor emits xt =\nd with probability \u03b7 and t with probability 1 \u2212 \u03b7. The transition out of start is evident as\nsoon as the \ufb01rst d or t is observed, while the magnitude of \u03b7 controls the \u201cconfusability\u201d of\nthe target and distractor states. Figure 2B shows a typical run from this generative model.\nThe transition into target happens on step 10 (top), and the outputs generated are a mixture\nof t and d(middle), with an overall prevalence of t (bottom).\n\nExact inference on this model can be performed in a manner similar to the forward pass in\na standard HMM:\n\nP (st|x1, . . . , xt) \u221d p(xt|st) X\n\nst\u22121\n\nP (st|st\u22121)P (st\u22121|x1, . . . , xt\u22121) .\n\n(2)\n\nBecause start does not produce outputs, as soon as the \ufb01rst t is observed, the probability of\nstart plummets to 0. There then ensues an inferential battle between target and distractor,\nwith the latter having the initial advantage, since its prior probability is 80%.\n\n\fA\n\n1.0\n\n1\u2212q(t)\n\n0.2 q(t)\n\n0.8 q(t)\n\nstart\n\noutputs\n\nT\n\nD\n\ntarget\n\n\u03b7\n1\u2212\u03b7\n1\u2212\u03b7\n\u03b7\n\ndistract\n\n1.0\n\nB\n\nt\nd\ns\nt\nd\ns\n20\n10\n0\n\nstate\n\noutput\n\ncumulative\noutputs\n\n20\n10\ntimestep\n\n30\n\nC\n\ny\nt\ni\nl\ni\n\nb\na\nb\no\nr\nP\n\n1\n0.5\n0\n1\n0.5\n0\n1\n0.5\n0\n\nP(start)\n\nP(distract)\n\nP(target)\n\n10\ntimestep\n\n20\n\n30\n\nD\n\ny\nt\ni\nv\ni\nt\nc\na\nE\nN\n\n5\n0\n5\n0\n5\n0\n5\n0\n\nhit stim\n\nresp\n\nfa\n\nmiss\n\ncr\n\n10\n\n20\ntimestep\n\n30\n\nFigure 2: The model. (A) Task is modeled as a hidden Markov model (HMM), with transi-\ntions from start to either distractor (probability .8) or target (probability .2). The transi-\ntions happen between timesteps 6 and 10 with uniform probability; distractor and target\nare absorbing states. The only outputs are from the absorbing states, and the two have over-\nlapping output distributions over t and d with probabilities \u03b7 > .5 for their \u201cown\u201d output (t\nfor target, and d for distractor), and 1\u2212 \u03b7 for the other output. (B) Sample run with a transi-\ntion from start to target at timestep 10 (upper). The outputs favor target once the state has\nchanged (middle), more clearly shown in the cumulative plot (bottom). (C) Correct prob-\nabilistic inference in the task leads to the probabilities for the three states as shown. The\ndistractor\u2019s initial advantage arises from a base rate effect, as it is the more likely default\ntransition. (D) Model NE signal for four trials including one for hit (top; same trials as in\nB;C), a false alarm (fa), a miss (miss) and a correct rejection (cr). The second vertical line\nrepresents the point at which the decision was taken (target vs. distractor).\n\nBecause of the preponderance of transitions to distractor over target, the distractor state\ncan be thought of as the reference or default state. Evidence against that default state is\na form of unexpected uncertainty within a task, and we propose that phasic NE reports\nthis uncertainty. More speci\ufb01cally, NE signals P (target|x1, . . . , xt)/P (target), where\nP (target) = .2 is the prior probability of observing a target trial. We assume that a\ntarget-response is initiated when P (st|x1, . . . , xt) exceeds 0.95, or equivalently, when\nthe NE signal exceeds 0.95/P (target). This implies the following intuitive relationship:\nthe smaller the probability of the non-default state target the greater the NE-mediated\n\u201csurprise\u201d signal has to be in order to convince the inferential system that an anomalous\nstimulus has been observed. We also assume that if the posterior probability of target\nreaches 0.01, then the trial ends with no action (either a cr or a miss). The asymmetry in\nthe thresholds arises from the asymmetry in the response contingencies of the task. Further,\nto model non-inferential errors, we assume that there is probability of 0.0005 per timestep\nof releasing the bar after the transition out of start. Once a decision is reached, the NE\nsignal is set back to baseline (1, for equal prior and posterior) after a delay of 5 timesteps.\nNote that the precise form of the mapping from unexpected uncertainty to NE spikes is\nrather arbitrary. In particular, there may be a strong non-linearity, such as a thresholded\nresponse pro\ufb01le. For simplicity, we assume a linear mapping between the two.\nThe NE activity during the start state is also rather arbitrary. Activity is at baseline before\nthe stimulus comes on, since prior and posterior match when there is no explicit information\nfrom the world. When the stimulus comes on, the divisive normalization makes the activity\ngo above baseline because although the transition was expected, its occurrence was not\npredicted with perfect precision. The magnitude of this activity depends on the precision\nof the model of the time of the transition; and the uncertainty in the interval timer. We set\nit to a small super-baseline level to match the data.\n\n\fA stim\n\n4\n\ny\nt\ni\nv\ni\nt\nc\na\nE\nN\n\n3\n\n2\n\n1\n\n0\n\nhit\n\nfa\n\nmiss\ncr\n\n10 20 30 40\n\nTimestep\n\nB\n\n5\n4\n3\n2\n1\n0\n\nresp\n\n10 20 30 40\n\nTimestep\n\nFigure 3: NE activity. (A) NE activity locked to the stimulus onset (ie the transition out of\nstart). (B) NE activity response-locked to the decision to act, just for hit and fa trials. Note\nthe difference in scale between the two \ufb01gures.\n\n3 Results\n\nFigure 2C illustrates the inferential performance of the model for the sample run in Fig-\nure 2B;C. When the \ufb01rst t is observed on timestep 10, the probability of start drops to 0 and\nthe probability of distractor, which has an initial advantage over target due to its higher\nprobability, eventually loses out to target as the evidence overwhelms the prior. Figure 2D\nshows the model\u2019s NE signal for one example each of hit, fa, miss, and cr trials.\n\nFigure 3 presents our main results. Figure 3A shows the average NE signal for the four\nclasses of responses (hit, false alarm, miss, and correct rejection), time-locked to the start\nof the stimulus. These traces should be compared with those in Figure 1E. The basic\nform of the rise of the signal in the model is broadly similar to that in the data; as we\nhave argued, the fall is rather arbitrary. Figure 3B shows the average signal locked to\nthe time of reaction (for hit and false alarm trials) rather than stimulus onset. As in the\ndata (Figure 1F), response-locked activities are much more tightly clustered, although this\n\ufb02atters the model somewhat, since we do not allow for any variability in the response time\nas a function of when the probability of state target reaches the threshold. Since the decay\nof the signal following a response is unconstrained, the trace terminates when the response\nis determined, usually when the probability of target reaches threshold, but also sometimes\nwhen there is an accidental erroneous response.\n\nFigure 4 shows some additional features of the NE signal in this case. Figure 4A compares\nthe effect of making the discrimination between target and distractor more or less dif\ufb01cult\nin the model (upper) and in the data (lower; [16]). As in the data, the stimulus-locked NE\nsignal is somewhat broader for the more dif\ufb01cult case, since information has to build up\nover a longer period. Also as in the data, correct rejections are much less affected than hits.\nFigure 4B shows response locked NE. Although it is correctly slightly broader for the more\ndif\ufb01cult discrimination, the timing is not quite the same. This is largely due to the lack of\na realistic model tying the defeat of the default state assumption to a behavioral response.\nFor the easy task (\u03b7 = 0.675), there were 19% hits, 1.5% false alarms, 1% misses and 77%\ncorrect rejections. For the dif\ufb01cult task (\u03b7 = 0.65) the main difference was an increase in\nthe number of misses to 1.5%, largely at the expense of hits. Note that since the NE signal\nis calculated relative to the prior likelihood, making target more likely would reduce the\nNE signal exactly proportionally. The data certainly hint at such a reduction [17] although\nthe precise proportionality is not clear.\n\n4 Discussion\n\nThe present model of the phasic activity of NE cells is a direct and major extension of\nour previous model of tonic aspects of this neuromodulator. The key difference is that\n\n\fB\n\nA\n\nc\ne\ns\n/\ns\ne\nk\np\nS\n\ni\n\nTime (sec)\n\nTime (sec)\n\nC\n\ny\nt\ni\nv\ni\nt\nc\na\nE\nN\n\n4\n\n3\n\n2\n\n1\n\n0\n\nD\n\n5\n\n4\n\n3\n\n2\n\n1\n\n0\n\n40\n\n10\n\n20\n\n30\n\nTimestep\n\n40\n\nhit\n\ncr\n\n10\n\n20\n\n30\n\nTimestep\n\nFigure 4: NE activities and task dif\ufb01culty. (A) Stimulus-locked LC responses are slower\nand broader for a more dif\ufb01cult discrimination; where dif\ufb01culty is controlled by the simi-\nlarity of target and distractor stimuli. (B) When aligned to response, LC activities for easy\nand dif\ufb01cult discriminations are more similar, although their response in the more dif\ufb01cult\ncondition is still somewhat attenuated compared to the easy one. Data in A;B adapted from\n[16]. (C) Discrimination dif\ufb01culty in the model is controlled by the parameter \u03b7. When \u03b7\nis reduced from 0.675 (easy; solid) to 0.65 (hard; dashed), simulated NE activity also be-\ncomes slower and broader when aligned to stimulus. (D) Same traces aligned to response\nindicate NE activity in the dif\ufb01cult condition is attenuated in the model.\n\nunexpected uncertainty is now about the state within a current characterization of the task\nrather than about the characterization as a whole. These aspects of NE functionality are\nlikely quite widespread, and allow us to account for a much wider range of data on this\nneuromodulator.\n\nIn the model, NE activity is explicitly normalized by prior probabilities arising from the\ndefault state transitions in tasks. This is necessary to measure speci\ufb01cally unexpected un-\ncertainty, and explains the decrement in NE phasic response as a function of the target\nprobability [17]. It is also associated with the small activation to the stimulus onset, al-\nthough the precise form of this deserves closer scrutiny. For instance, if subjects were to\nbuild a richer model of the statistics of the time of the transition out of the start state, then\nwe should see this re\ufb02ected directly in the NE signal even before the stimulus comes on,\nfor instance related to the inverse of the survival function for the transition. We would also\nexpect this transition to effect a different NE signature if stimuli were expected during start\nthat could also be confused with those expected during target and distractor.\n\nIf NE indeed reports on the failure of the current state within the model of the task to\naccount successfully for the observations, then what effect should it have? One useful\nway to think about the signal is in terms of an interrupt signal in computers. In these,\na control program establishes a set of conditions (eg keyboard input) under which nor-\nmal processing should be interrupted, in order that the consequence of the interrupt (eg a\nkeystroke) can be appropriately handled. Computers have highly centralized processing\narchitecture, and therefore the interrupt signal only needs a very limited spatial extent to\nexert a widespread effect on the course of computation. By contrast, processing in the\nbrain is highly distributed, and therefore it is necessary for the interrupt signal to have a\nwidespread distribution, so that the full rami\ufb01cations of the failure of the current state can\nbe felt. Neuromodulatory systems are ideal vehicles for the signal.\n\nThe interrupt signal should engage mechanisms for establishing the new state, which then\nallows a new set of conditions to be established as to which interrupts will be allowed to\noccur, and also to take any appropriate action (as in the task we modeled). The interrupt\nsignal can be expected to be bene\ufb01cial, for instance, when there is competition between\ntasks for the use of neural resources such as receptive \ufb01elds [8].\n\nApart from interrupts such as these under sophisticated top-down control, there are also\nmore basic contingencies from things such as critical potential threats and stressors that\n\n\fshould exert a rapid and dramatic effect on neural processing (these also have computa-\ntional analogues in signals such as that power is about to fail). The NE system is duly\nsubject to what might be considered as bottom-up as well as top-down in\ufb02uences [21].\n\nThe interrupt-based account is a close relative of existing notions of phasic NE. For in-\nstance, NE has been implicated in the process of alerting [23]. The difference from our\naccount is perhaps the stronger tie in the latter to actual behavioral output. A task with\nsecond-order contingencies may help to differentiate the two accounts. There are also\nclose relations with theories [20, 5] that suggest how NE may be an integral part of an op-\ntimal decisional strategy. These propose that NE controls the gain in competitive decision-\nmaking networks that implement sequential decision-making [22], essentially by reporting\non the changes in the statistical structure of the inputs induced by stimulus onset. It is also\nsuggested that a more extreme change in the gain, destabilizing the competitive networks\nthrough explosive symmetry breaking, can be used to freeze or lock-in any small difference\nin the competing activities.\n\nThe idea that NE can signal the change in the input statistics occasioned by the (temporally-\nunpredictable) occurrence of the target is highly appealing. However, the statistics of the\ninput change when either the target or the distractor appears, and so the preference for\nresponding to the target at the expense of the distractor is strange. The effect of forcing the\ndecision making network to become unstable, and therefore enforcing a speeded decision\nis much closer to an interrupt; but then it is not clear why this signal should decrease as\nthe target becomes more common. Further, since in the unstable regime, the statistical\noptimality of integration is effectively abandoned, the computational appeal of the signal\nis somewhat weakened. However, this alternative theory does make an important link to\nsequential statistical analysis [22], raising issues about things like thresholds for deciding\ntarget and distractor that should be important foci of future work here too.\n\nFigure 1C shows an additional phenomenon that has arisen in a task when subjects were not\neven occasionally taxed with dif\ufb01cult discrimination problems. The overall performance\n\ufb02uctuates dramatically (shown by the changing false alarm rate), in a manner that is tightly\ncorrelated with \ufb02uctuations in tonic NE activity. Periods of high tonic activity are corre-\nlated with low phasic activation to the targets (data not shown). Aston-Jones, Cohen and\ntheir colleagues [20, 3] have suggested that NE regulates the balance between exploration\nand exploitation. The high tonic phase is associated with the former, with subjects failing\nto concentrate on the contingencies that lead to their current rewards in order to search\nfor stimuli or actions that might be associated with better rewards.\nIncreasing the ease\nof interruptability to either external cues or internal state changes, could certainly lead to\napparently exploratory behavior. However, there is little evidence as to how this sort of\nexploration is being actively determined, since, for instance, the macroscopic \ufb02uctuations\nevident in Figure 1C do not arise in response to any experimental contingency. Given the\nrelationship between phasic and tonic \ufb01ring, further investigation of these periodic \ufb02uctua-\ntions and their implications would be desirable.\n\nFinally, in our previous model [24, 25], tonic NE was closely coupled with tonic acetyl-\ncholine (ACh), with the latter reporting expected rather than unexpected uncertainty. The\naccount of ACh should transfer somewhat directly into the short-term contingencies within\na task \u2013 we might expect it to be involved in reporting on aspects of the known variability\nassociated with each state, including each distinct stimulus state as well as the no-stimulus\nstate. As such, this ACh signal might be expected to be relatively more tonic than NE (an\neffect that is also apparent in our previous work on more tonic interactions between ACh\nand NE (eg Figure 2 of [24]). One attractive target for an account along these lines is the\nsustained attention task studied by Sarter and colleagues, which involves temporal uncer-\ntainty. Performance in this task is exquisitely sensitive to cholinergic manipulation [19],\nbut unaffected by gross noradrenergic manipulation [15]. We may again expect there to be\ninteresting part-opponent and part-synergistic interactions between the neuromodulators.\n\n\fAcknowledgements\n\nWe are grateful to Gary Aston-Jones, Sebastien Bouret, Jonathan Cohen, Peter Latham,\nSusan Sara, and Eric Shea-Brown for helpful discussions. Funding was from the Gatsby\nCharitable Foundation, the EU BIBA project and the ACI Neurosciences Int\u00b4egratives et\nComputationnelles of the French Ministry of Research.\n\nReferences\n[1] Aston-Jones, G, Rajkowski, J, Kubiak, P & Alexinsky, T (1994). Locus coeruleus neurons in\nmonkey are selectively activated by attended cues in a vigilance task. J. Neurosci. 14:4467-\n4480.\n\n[2] Aston-Jones, G, Rajkowski, J & Kubiak, P (1997). Conditioned responses of monkey locus\ncoeruleus neurons anticipate Acquisition of discriminative behavior in a vigilance task. Neuro-\nscience 80:697-715.\n\n[3] Aston-Jones, G, Rajkowski, J & Cohen, J (2000). Locus coeruleus and regulation of behavioral\n\n\ufb02exibility and attention. Prog. Brain Res. 126:165-182.\n\n[4] Bouret, S & Sara, SJ (2004). Reward expectation, orientation of attention and locus coeruleus-\n\nmedial frontal cortex interplay during learning. Eur. J. Neurosci. 20:791-802.\n\n[5] Brown, E, Gao, J, Holmes, P, Bogacz, R, Gilzenrat, M & Cohen, JD (2005). Simple neural\n\nnetworks that optimize decisions. Int. J. Bif. & Chaos, in press.\n\n[6] David Johnson, J (2003). Noradrenergic control of cognition: global attenuation and an inter-\n\nrupt function. Med. Hypoth. 60:689-692.\n\n[7] Dayan, P & Yu, AJ (2001). ACh, uncertainty, and cortical inference. NIPS 2001.\n[8] Desimone, R & Duncan, J (1995). Neural mechanisms of selective visual attention. Annual\n\nNIPS 2002.\n\n[25] Yu, AJ & Dayan, P (2005). Uncertainty, neuromodulation, and attention. Neuron 46, 681-692.\n\n[9] Devauges, V & Sara, SJ (1990). Activation of the noradrenergic system facilitates an attentional\n\nReviews in Neuroscience 18:193-222.\n\nshift in the rat. Beh. Brain Res. 39:19-28.\n\n[10] Doya, K (2002). Metalearning and neuromodulation. Neur. Netw. 15:495-506.\n[11] Foote, SL, Freedman, R & Oliver, AP (1975). Effects of putative neurotransmitters on neuronal\n\nactivity in monkey auditory cortex. Brain Res. 86:229-242.\n\n[12] Freedman, R, Foote, SL & Bloom, FE (1975) Histochemical characterization of a neocortical\nprojection of the nucleus locus coeruleus in the squirrel monkey. J. Comp. Neurol. 164:209-231.\n\n[13] Jouvet, M (1969). Biogenic amines and the states of sleep. Science 163:32-41.\n[14] Marder, E & Thirumalai, V (2002). Cellular, synaptic and network effects of neuromodulation.\n\nNeur. Netw. 15:479-493.\n\n[15] McGaughy, J, Sandstrom, M, Ruland, S, Bruno JP & Sarter, M (1997). Lack of effects of lesions\n\nof the dorsal noradrenergic bundle on behavioral vigilance. Beh. Neurosci. 111:646-652.\n\n[16] Rajkowski, J, Majczynski, H, Clayton, E & Aston-Jones, G (2004). Activation of monkey\nlocus coeruleus neurons varies with dif\ufb01culty and performance in a target detection task. J.\nNeurophysiol. 92:361-371.\n\n[17] Rajkowski, J, Majczynski, H, Clayton, E, Cohen, JD & Aston-Jones, G (2002). Phasic acti-\nvation of monkey locus coeruleus (LC) neurons with recognition of motivationally relevant\nstimuli. Society for Neuroscience, Abstracts 86.10.\n\n[18] Sara, SJ & Segal, M (1991). Plasticity of sensory responses of locus coeruleus neurons in the\n\nbehaving rat: implications for cognition. Prog. Brain Res. 88:571-585.\n\n[19] Turchi, J & Sarter, M (2001). Bidirectional modulation of basal forebrain NMDA receptor\nfunction differentially affects visual attention but not visual discrimination performance. Neu-\nroscience 104:407-417.\n\n[20] Usher, M, Cohen, JD, Servan-Schreiber, D, Rajkowski, J & Aston-Jones, G (1999). The role of\n\nlocus coeruleus in the regulation of cognitive performance. Science 283:549-554.\n\n[21] Van Bockstaele, EJ, Chan, J & Pickel, VM (1996). Input from central nucleus of the amygdala\nefferents to pericoerulear dendrites, some of which contain tyrosine hydroxylase immunoreac-\ntivity. Journal of Neuroscience Research 45:289-302.\n\n[22] Wald, A (1947). Sequential Analysis. New York, NY: John Wiley & Sons.\n[23] Witte, EA & Marrocco, RT (1997). Alteration of brain noradrenergic activity in rhesus monkeys\n\naffects the alerting component of covert orienting. Psychopharmacology 132:315-323.\n\n[24] Yu, AJ & Dayan, P (2003). Expected and unexpected uncertainty. ACh and NE in the neocortex.\n\n\f", "award": [], "sourceid": 2752, "authors": [{"given_name": "Peter", "family_name": "Dayan", "institution": null}, {"given_name": "Angela", "family_name": "Yu", "institution": null}]}