{"title": "Sensory Adaptation within a Bayesian Framework for Perception", "book": "Advances in Neural Information Processing Systems", "page_first": 1289, "page_last": 1296, "abstract": "", "full_text": "Sensory Adaptation within a Bayesian\n\nFramework for Perception\n\nAlan A. Stocker(cid:3) and Eero P. Simoncelli\n\nHoward Hughes Medical Institute and\n\nCenter for Neural Science\n\nNew York University\n\nAbstract\n\nWe extend a previously developed Bayesian framework for perception\nto account for sensory adaptation. We \ufb01rst note that the perceptual ef-\nfects of adaptation seems inconsistent with an adjustment of the inter-\nnally represented prior distribution. Instead, we postulate that adaptation\nincreases the signal-to-noise ratio of the measurements by adapting the\noperational range of the measurement stage to the input range. We show\nthat this changes the likelihood function in such a way that the Bayesian\nestimator model can account for reported perceptual behavior. In particu-\nlar, we compare the model\u2019s predictions to human motion discrimination\ndata and demonstrate that the model accounts for the commonly observed\nperceptual adaptation effects of repulsion and enhanced discriminability.\n\n1 Motivation\n\nA growing number of studies support the notion that humans are nearly optimal when per-\nforming perceptual estimation tasks that require the combination of sensory observations\nwith a priori knowledge. The Bayesian formulation of these problems de\ufb01nes the optimal\nstrategy, and provides a principled yet simple computational framework for perception that\ncan account for a large number of known perceptual effects and illusions, as demonstrated\nin sensorimotor learning [1], cue combination [2], or visual motion perception [3], just to\nname a few of the many examples.\n\nAdaptation is a fundamental phenomenon in sensory perception that seems to occur at all\nprocessing levels and modalities. A variety of computational principles have been sug-\ngested as explanations for adaptation. Many of these are based on the concept of maximiz-\ning the sensory information an observer can obtain about a stimulus despite limited sensory\nresources [4, 5, 6]. More mechanistically, adaptation can be interpreted as the attempt of\nthe sensory system to adjusts its (limited) dynamic range such that it is maximally infor-\nmative with respect to the statistics of the stimulus. A typical example is observed in the\nretina, which manages to encode light intensities that vary over nine orders of magnitude\nusing ganglion cells whose dynamic range covers only two orders of magnitude. This is\nachieved by adapting to the local mean as well as higher order statistics of the visual input\nover short time-scales [7].\n\n(cid:3)corresponding author.\n\n\fIf a Bayesian framework is to provide a valid computational explanation of perceptual\nprocesses, then it needs to account for the behavior of a perceptual system, regardless of\nits adaptation state. In general, adaptation in a sensory estimation task seems to have two\nfundamental effects on subsequent perception:\n\n(cid:15) Repulsion: The estimate of parameters of subsequent stimuli are repelled by\nthose of the adaptor stimulus, i.e. the perceived values for the stimulus variable\nthat is subject to the estimation task are more distant from the adaptor value after\nadaptation. This repulsive effect has been reported for perception of visual speed\n(e.g. [8, 9]), direction-of-motion [10], and orientation [11].\n\n(cid:15) Increased sensitivity: Adaptation increases the observer\u2019s discrimination ability\naround the adaptor (e.g. for visual speed [12, 13]), however it also seems to de-\ncrease it further away from the adaptor as shown in the case of direction-of-motion\ndiscrimination [14].\n\nIn this paper, we show that these two perceptual effects can be explained within a Bayesian\nestimation framework of perception. Note that our description is at an abstract functional\nlevel - we do not attempt to provide a computational model for the underlying mechanisms\nresponsible for adaptation, and this clearly separates this paper from other work which\nmight seem at \ufb01rst glance similar [e.g., 15].\n\n2 Adaptive Bayesian estimator framework\n\nSuppose that an observer wants to estimate a property of a stimulus denoted by the variable\n(cid:18), based on a measurement m. In general, the measurement can be vector-valued, and\nis corrupted by both internal and external noise. Hence, combining the noisy information\ngained by the measurement m with a priori knowledge about (cid:18) is advantageous. According\nto Bayes\u2019 rule\n\np((cid:18)jm) =\n\n1\n(cid:11)\n\np(mj(cid:18))p((cid:18)) :\n\n(1)\n\nThat is, the probability of stimulus value (cid:18) given m (posterior) is the product of the likeli-\nhood p(mj(cid:18)) of the particular measurement and the prior p((cid:18)). The normalization constant\n(cid:11) serves to ensure that the posterior is a proper probability distribution. Under the assump-\ntion of a squared-error loss function, the optimal estimate ^(cid:18)(m) is the mean of the posterior,\nthus\n\n^(cid:18)(m) = Z 1\n\n0\n\n(cid:18) p((cid:18)jm) d(cid:18) :\n\n(2)\n\nNote that ^(cid:18)(m) describes an estimate for a single measurement m. As discussed in [16],\nthe measurement will vary stochastically over the course of many exposures to the same\nstimulus, and thus the estimator will also vary. We return to this issue in Section 3.2.\n\nFigure 1a illustrates a Bayesian estimator, in which the shape of the (arbitrary) prior dis-\ntribution leads on average to a shift of the estimate toward a lower value of (cid:18) than the true\nstimulus value (cid:18)stim. The likelihood and the prior are the fundamental constituents of the\nBayesian estimator model. Our goal is to describe how adaptation alters these constituents\nso as to account for the perceptual effects of repulsion and increased sensitivity.\n\nAdaptation does not change the prior ...\n\nAn intuitively sensible hypothesis is that adaptation changes the prior distribution. Since\nthe prior is meant to re\ufb02ect the knowledge the observer has about the distribution of occur-\nrences of the variable (cid:18) in the world, repeated viewing of stimuli with the same parameter\n\n\fa\n\ny\nt\ni\nl\ni\n\nb\na\nb\no\nr\np\n\nposterior\n\nprior\n\n\u00c3\n\nlikelihood\n\nattraction !\n\nb\n\ny\nt\ni\nl\ni\n\nb\na\nb\no\nr\np\n\nmodified prior\n\n'\u02c6q\n\nq\n\nadapt\n\nq\n\nFigure 1: Hypothetical model in which adaptation alters the prior distribution. a) Un-\nadapted Bayesian estimation con\ufb01guration in which the prior leads to a shift of the estimate\n^(cid:18), relative to the stimulus parameter (cid:18)stim. Both the likelihood function and the prior distri-\nbution contribute to the exact value of the estimate ^(cid:18) (mean of the posterior). b) Adaptation\nacts by increasing the prior distribution around the value, (cid:18)adapt, of the adapting stimulus\nparameter. Consequently, an subsequent estimate ^(cid:18)0 of the same stimulus parameter value\n(cid:18)stim is attracted toward the adaptor. This is the opposite of observed perceptual effects,\nand we thus conclude that adjustments of the prior in a Bayesian model do not account for\nadaptation.\n\nvalue (cid:18)adapt should presumably increase the prior probability in the vicinity of (cid:18)adapt. Fig-\nure 1b schematically illustrates the effect of such a change in the prior distribution. The\nestimated (perceived) value of the parameter under the adapted condition is attracted to the\nadapting parameter value. In order to account for observed perceptual repulsion effects,\nthe prior would have to decrease at the location of the adapting parameter, a behavior that\nseems fundamentally inconsistent with the notion of a prior distribution.\n\n... but increases the reliability of the measurements\n\nSince a change in the prior distribution is not consistent with repulsion, we are led to the\nconclusion that adaptation must change the likelihood function. But why, and how should\nthis occur?\n\nIn order to answer this question, we reconsider the functional purpose of adaptation. We as-\nsume that adaptation acts to allocate more resources to the representation of the parameter\nvalues in the vicinity of the adaptor [4], resulting in a local increase in the signal-to-noise\nratio (SNR). This can be accomplished, for example, by dynamically adjusting the opera-\ntional range to the statistics of the input. This kind of increased operational gain around\nthe adaptor has been effectively demonstrated in the process of retinal adaptation [17]. In\nthe context of our Bayesian estimator framework, and restricting to the simple case of a\nscalar-valued measurement, adaptation results in a narrower conditional probability den-\nsity p(mj(cid:18)) in the immediate vicinity of the adaptor, thus an increase in the reliability of\nthe measurement m. This is offset by a broadening of the conditional probability den-\nsity p(mj(cid:18)) in the region beyond the adaptor vicinity (we assume that total resources are\nconserved, and thus an increase around the adaptor must necessarily lead to a decrease\nelsewhere).\n\nFigure 2 illustrates the effect of this local increase in signal-to-noise ratio on the likeli-\n\nq\nq\n\funadapted\n\nadapted\n\n1/SNR\n\nq\n\nadapt\n\np(m | )\nq\n\n2 \n\n'\n\nq\n\nq\n\np(m | )2\nq\n\nq\n\n1\n\nq\n\n2\n\nq\n\n1\n\n2q\n\nq\n\nq\n\nm\n\nq\n\n'\np(m | )\nq\n\n1\n\np(m | )1\nq\n\nq\n\nadapt\n\np(m| )2\nq\n\n'q\np(m| )2\n\nlikelihoods\n\nm2\n\nm1\n\nm\n\nq\n\np(m| ) \n\nq\n1\n\n'q\np(m| )1\n\n'\np(m| )\nadapt\n\nq\n\nconditionals\n\nFigure 2: Measurement noise, conditionals and likelihoods. The two-dimensional condi-\ntional density, p(mj(cid:18)), is shown as a grayscale image for both the unadapted and adapted\ncases. We assume here that adaptation increases the reliability (SNR) of the measurement\naround the parameter value of the adaptor. This is balanced by a decrease in SNR of the\nmeasurement further away from the adaptor. Because the likelihood is a function of (cid:18) (hor-\nizontal slices, shown plotted at right), this results in an asymmetric change in the likelihood\nthat is in agreement with a repulsive effect on the estimate.\n\n\fa\n\n+\n\n \n \n \n\n^\nq\nD\n\n0\n\n-\n\nb\n\n]\ng\ne\nd\n \n[\n \n \n\n^\nq\nD\n\n60\n\n30\n\n0\n\n-30\n\n-60\n\n-180\n\n-90\n\n \nq\n\nadapt\n\n90\n\n180\nq [deg]\n\nq\n\nq\n \n\nadapt\n\nFigure 3: Repulsion: Model predictions vs. human psychophysics. a) Difference in per-\nceived direction in the pre- and post-adaptation condition, as predicted by the model. Post-\nadaptive percepts of motion direction are repelled away from the direction of the adaptor.\nb) Typical human subject data show a qualitatively similar repulsive effect. Data (and \ufb01t)\nare replotted from [10].\n\nhood function. The two gray-scale images represent the conditional probability densities,\np(mj(cid:18)), in the unadapted and the adapted state. They are formed by assuming additive\nnoise on the measurement m of constant variance (unadapted) or with a variance that\ndecreases symmetrically in the vicinity of the adaptor parameter value (cid:18)adapt, and grows\nslightly in the region beyond. In the unadapted state, the likelihood is convolutional and\nthe shape and variance are equivalent to the distribution of measurement noise. However,\nin the adapted state, because the likelihood is a function of (cid:18) (horizontal slice through the\nconditional surface) it is no longer convolutional around the adaptor. As a result, the mean\nis pushed away from the adaptor, as illustrated in the two graphs on the right. Assuming\nthat the prior distribution is fairly smooth, this repulsion effect is transferred to the posterior\ndistribution, and thus to the estimate.\n\n3 Simulation Results\n\nWe have qualitatively demonstrated that an increase in the measurement reliability around\nthe adaptor is consistent with the repulsive effects commonly seen as a result of percep-\ntual adaptation. In this section, we simulate an adapted Bayesian observer by assuming a\nsimple model for the changes in signal-to-noise ratio due to adaptation. We address both\nrepulsion and changes in discrimination threshold. In particular, we compare our model\npredictions with previously published data from psychophysical experiments examining\nhuman perception of motion direction.\n\n3.1 Repulsion\n\nIn the unadapted state, we assume the measurement noise to be additive and normally\ndistributed, and constant over the whole measurement space. Thus, assuming that m and\n(cid:18) live in the same space, the likelihood is a Gaussian of constant width. In the adapted\nstate, we assume a simple functional description for the variance of the measurement noise\naround the adapter. Speci\ufb01cally, we use a constant plus a difference of two Gaussians,\n\n\fa\n\nb\n\n1.8\n\n \n\n \n\nl\n\nd\no\nh\ns\ne\nr\nh\nt\n \nn\no\ni\nt\na\nn\nm\n\ni\n\n \n\ni\nr\nc\ns\ni\nd\ne\nv\ni\nt\na\ne\nr\n\nl\n\n1\n\nl\n\nd\no\nh\ns\ne\nr\nh\nt\n \nn\no\ni\nt\na\nn\nm\n\ni\n\n \n\ni\nr\nc\ns\ni\nd\ne\nv\ni\nt\na\ne\nr\n\nl\n\n1.6\n\n1.4\n\n1.2\n\n1\n\n0.8\n\nq\n\nadapt\n\nq\n\n-40\n\n-20\n\n \nq\n\nadapt\n\n20\n\n40\n\nq [deg]\n\nFigure 4: Discrimination thresholds: Model predictions vs. human psychophysics. a) The\nmodel predicts that thresholds for direction discrimination are reduced at the adaptor. It\nalso predicts two side-lobes of increased threshold at further distance from the adaptor.\nb) Data of human psychophysics are in qualitative agreement with the model. Data are\nreplotted from [14] (see also [11]).\n\neach having equal area, with one twice as broad as the other (see Fig. 2).\nFinally, for\nsimplicity, we assume a \ufb02at prior, but any reasonable smooth prior would lead to results\nthat are qualitatively similar. Then, according to (2) we compute the predicted estimate of\nmotion direction in both the unadapted and the adapted case.\n\nFigure 3a shows the predicted difference between the pre- and post-adaptive average esti-\nmate of direction, as a function of the stimulus direction, (cid:18)stim. The adaptor is indicated with\nan arrow. The repulsive effect is clearly visible. For comparison, Figure 3b shows human\nsubject data replotted from [10]. The perceived motion direction of a grating was estimated,\nunder both adapted and unadapted conditions, using a two-alternative-forced-choice exper-\nimental paradigm. The plot shows the change in perceived direction as a function of test\nstimulus direction relative to that of the adaptor. Comparison of the two panels of Figure 3\nindicate that despite the highly simpli\ufb01ed construction of the model, the prediction is quite\ngood, and even includes the small but consistent repulsive effects observed 180 degrees\nfrom the adaptor.\n\n3.2 Changes in discrimination threshold\n\nAdaptation also changes the ability of human observers to discriminate between the di-\nrection of two different moving stimuli. In order to model discrimination thresholds, we\nneed to consider a Bayesian framework that can account not only for the mean of the es-\ntimate but also its variability. We have recently developed such a framework, and used\nit to quantitatively constrain the likelihood and the prior from psychophysical data [16].\nThis framework accounts for the effect of the measurement noise on the variability of the\nestimate ^(cid:18). Speci\ufb01cally, it provides a characterization of the distribution p( ^(cid:18)j(cid:18)stim) of the\nestimate for a given stimulus direction in terms of its expected value and its variance as a\nfunction of the measurement noise. As in [16] we write\n@ ^(cid:18)(m)\n\nvarh^(cid:18)j(cid:18)stimi = varhmi(\n\n)2jm=(cid:18)stim :\n\n@m\n\n(3)\n\nAssuming that discrimination threshold is proportional\n\nto the standard deviation,\n\n\fqvarh^(cid:18)j(cid:18)stimi, we can now predict how discrimination thresholds should change after adap-\n\ntation. Figure 4a shows the predicted change in discrimination thresholds relative to the un-\nadapted condition for the same model parameters as in the repulsion example (Figure 3a).\nThresholds are slightly reduced at the adaptor, but increase symmetrically for directions\nfurther away from the adaptor. For comparison, Figure 4b shows the relative change in dis-\ncrimination thresholds for a typical human subject [14]. Again, the behavior of the human\nobserver is qualitatively well predicted.\n\n4 Discussion\n\nWe have shown that adaptation can be incorporated into a Bayesian estimation framework\nfor human sensory perception. Adaptation seems unlikely to manifest itself as a change\nin the internal representation of prior distributions, as this would lead to perceptual bias\neffects that are opposite to those observed in human subjects. Instead, we argue that adap-\ntation leads to an increase in reliability of the measurement in the vicinity of the adapting\nstimulus parameter. We show that this change in the measurement reliability results in\nchanges of the likelihood function, and that an estimator that utilizes this likelihood func-\ntion will exhibit the commonly-observed adaptation effects of repulsion and changes in\ndiscrimination threshold. We further con\ufb01rm our model by making quantitative predictions\nand comparing them with known psychophysical data in the case of human perception of\nmotion direction.\n\nMany open questions remain. The results demonstrated here indicate that a resource alloca-\ntion explanation is consistent with the functional effects of adaptation, but it seems unlikely\nthat theory alone can lead to a unique quantitative prediction of the detailed form of these\neffects. Speci\ufb01cally, the constraints imposed by biological implementation are likely to\nplay a role in determining the changes in measurement noise as a function of adaptor pa-\nrameter value, and it will be important to characterize and interpret neural response changes\nin the context of our framework. Also, although we have argued that changes in the prior\nseem inconsistent with adaptation effects, it may be that such changes do occur but are\noffset by the likelihood effect, or occur only on much longer timescales.\n\nLast, if one considers sensory perception as the result of a cascade of successive processing\nstages (with both feedforward and feedback connections), it becomes necessary to expand\nthe Bayesian description to describe this cascade [e.g., 18, 19]. For example, it may be\npossible to interpret this cascade as a sequence of Bayesian estimators, in which the mea-\nsurement of each stage consists of the estimate computed at the previous stage. Adaptation\ncould potentially occur in each of these processing stages, and it is of fundamental interest\nto understand how such a cascade can perform useful stable computations despite the fact\nthat each of its elements is constantly readjusting its response properties.\n\nReferences\n\n[1] K. K\u00a8ording and D. Wolpert.\n\n427(15):244\u2013247, January 2004.\n\nBayesian integration in sensorimotor learning. Nature,\n\n[2] D C Knill and W Richards, editors. Perception as Bayesian Inference. Cambridge University\n\nPress, 1996.\n\n[3] Y. Weiss, E. Simoncelli, and E. Adelson. Motion illusions as optimal percept. Nature Neuro-\n\nscience, 5(6):598\u2013604, June 2002.\n\n[4] H.B. Barlow. Vision: Coding and Ef\ufb01ciency, chapter A theory about the functional role and\nsynaptic mechanism of visual after-effects, pages 363\u2013375. Cambridge University Press., 1990.\n[5] M.J. Wainwright. Visual adaptation as optimal information transmission. Vision Research,\n\n39:3960\u20133974, 1999.\n\n\f[6] N. Brenner, W. Bialek, and R. de Ruyter van Steveninck. Adaptive rescaling maximizes infor-\n\nmation transmission. Neuron, 26:695\u2013702, June 2000.\n\n[7] S.M. Smirnakis, M.J. Berry, D.K. Warland, W. Bialek, and M. Meister. Adaptation of retinal\n\nprocessing to image contrast and spatial scale. Nature, 386:69\u201373, March 1997.\n\n[8] P. Thompson. Velocity after-effects: the effects of adaptation to moving stimuli on the percep-\n\ntion of subsequently seen moving stimuli. Vision Research, 21:337\u2013345, 1980.\n\n[9] A.T. Smith. Velocity coding: evidence from perceived velocity shifts. Vision Research,\n\n25(12):1969\u20131976, 1985.\n\n[10] P. Schrater and E. Simoncelli. Local velocity representation: evidence from motion adaptation.\n\nVision Research, 38:3899\u20133912, 1998.\n\n[11] C.W. Clifford. Perceptual adaptation: motion parallels orientation. Trends in Cognitive Sci-\n\nences, 6(3):136\u2013143, March 2002.\n\n[12] C. Clifford and P. Wenderoth. Adaptation to temporal modulaton can enhance differential speed\n\nsensitivity. Vision Research, 39:4324\u20134332, 1999.\n\n[13] A. Kristjansson. Increased sensitivity to speed changes during adaptation to \ufb01rst-order, but not\n\nto second-order motion. Vision Research, 41:1825\u20131832, 2001.\n\n[14] R.E. Phinney, C. Bowd, and R. Patterson. Direction-selective coding of stereoscopic (cyclo-\n\npean) motion. Vision Research, 37(7):865\u2013869, 1997.\n\n[15] N.M. Grzywacz and R.M. Balboa. A Bayesian framework for sensory adaptation. Neural\n\nComputation, 14:543\u2013559, 2002.\n\n[16] A.A. Stocker and E.P. Simoncelli. Constraining a Bayesian model of human visual speed per-\nception. In Lawrence K. Saul, Yair Weiss, and L\u00b4eon Bottou, editors, Advances in Neural Infor-\nmation Processing Systems NIPS 17, pages 1361\u20131368, Cambridge, MA, 2005. MIT Press.\n\n[17] D. Tranchina, J. Gordon, and R.M. Shapley. Retinal light adaptation \u2013 evidence for a feedback\n\nmechanism. Nature, 310:314\u2013316, July 1984.\n\n[18] S. Deneve. Bayesian inference in spiking neurons. In Lawrence K. Saul, Yair Weiss, and L\u00b4eon\nBottou, editors, Adv. Neural Information Processing Systems (NIPS*04), vol 17, Cambridge,\nMA, 2005. MIT Press.\n\n[19] R. Rao. Hierarchical Bayesian inference in networks of spiking neurons. In Lawrence K. Saul,\nYair Weiss, and L\u00b4eon Bottou, editors, Adv. Neural Information Processing Systems (NIPS*04),\nvol 17, Cambridge, MA, 2005. MIT Press.\n\n\f", "award": [], "sourceid": 2912, "authors": [{"given_name": "Alan", "family_name": "Stocker", "institution": null}, {"given_name": "Eero", "family_name": "Simoncelli", "institution": null}]}