{"title": "Eye Micro-movements Improve Stimulus Detection Beyond the Nyquist Limit in the Peripheral Retina", "book": "Advances in Neural Information Processing Systems", "page_first": 1475, "page_last": 1482, "abstract": "", "full_text": "Eye micro-movements improve stimulus\ndetection beyond the Nyquist limit in the\n\nperipheral retina\n\nMatthias H. Hennig and Florentin W\u00a8org\u00a8otter\n\nComputational Neuroscience\n\nPsychology\n\nUniversity of Stirling\nFK9 4LR Stirling, UK\n\n{hennig,worgott}@cn.stir.ac.uk\n\nAbstract\n\nEven under perfect \ufb01xation the human eye is under steady motion\n(tremor, microsaccades, slow drift). The \u201cdynamic\u201d theory of vi-\nsion [1, 2] states that eye-movements can improve hyperacuity. Accord-\ning to this theory, eye movements are thought to create variable spatial\nexcitation patterns on the photoreceptor grid, which will allow for better\nspatiotemporal summation at later stages. We reexamine this theory us-\ning a realistic model of the vertebrate retina by comparing responses of a\nresting and a moving eye. The performance of simulated ganglion cells\nin a hyperacuity task is evaluated by ideal observer analysis. We \ufb01nd that\nin the central retina eye-micromovements have no effect on the perfor-\nmance. Here optical blurring limits vernier acuity. In the retinal periph-\nery however, eye-micromovements clearly improve performance. Based\non ROC analysis, our predictions are quantitatively testable in electro-\nphysiological and psychophysical experiments.\n\n1 Introduction\nNormal visual acuity is limited by the photoreceptor distance on the retina to about 10 of\nvisual angle, which is imposed by the neural nyquist sampling limit. The human visual\nsystem, however, is capable of resolving certain stimuli (e.g. vernier stimuli) at a much\nhigher resolution of < 500. This effect, called hyperactuity, has given rise to a large number\nof psychophysical studies and several qualitative theories about perception as well as the\nunderlying neuronal properties. Most notably are the so-called \u201cdynamic\u201d and \u201cstatic\u201d\ntheories of vision [3], which claim that hyperacuity would require eye-micromovements\n(microtremor, microsaccades) or not. Along the dynamic theory it has been suggested by\nAverill and Weymouth [1] and later by Marshall and Talbot [2] that small eye-movements\nwould shift the photoreceptor grid across the stimulus leading to a better discriminability\nwhen appropriate spatiotemporal integration is used.\n\nIn a previous study we had designed a realistic and detailed model of the vertebrate\nretina [4]. This allows us for the \ufb01rst time to quantitatively test the Marshall-Talbot\n\n\fFigure 1: Overview of the model. A, Structure of the retina model. Photoreceptors (P)\nconnect to horizontal (H) and bipolar cells (B). Horizontal cells antagonize bipolar cells.\nBipolar cells provide the center input to ganglion cells (G) and the surround is mediated by a\nType 1 (1) amacrine cell [4]. B, Scaling of optical point spread functions (top curves), pho-\ntoreceptor (upper lines, values shown, data from [5]) and ganglion cell separation (lower\nlines, values shown, data from [6, 7]) at different retinal eccentricities. PSF\u2019s are shown for\nthe constant (straight lines) and scaled case (dashed lines). C, Spatial layout of the stimulus\n(S) and the photoreceptor (P) and ganglion cell (G) grids. D, Nyquist frequencies for pho-\ntoreceptors, P ganglion cells and the scaled PSF as a function of the eccentricity. Aliasing\noccurs in the shaded region.\n\ntheory under different experimental conditions. We will show that the presence of eye-\nmicromovements indeed improves hyperacuity. Contrary to earlier assumptions we \ufb01nd\nthat eye micromovements have no effect in the central part of the retina, where optical\nblurring de\ufb01nes the limit for hyperacuity tasks. At above 5\u25e6 retinal eccentricity, eye-\nmicromovements are clearly improving hyperacuity. Our approach relies on a model free\n(receiver-operator characteristic, ROC) analysis, and the reported results should be directly\nmeasurable in retinal ganglion cells and psychophysically.\n\n2 MATERIALS AND METHODS\n\nThe model used in this study is based on a previously described model of the light adapted\nretina. In this section, we only mention aspects which are important in the context of this\nstudy. For a detailed discussion of the model, see [4].\n\nBrie\ufb02y, the model consists of cone photoreceptors, horizontal and bipolar, amacrine and\nganglion cells (Fig. 1A). Neurons are arranged on homogeneous two-dimensional hexago-\n\n\fnal grids (Fig. 1C). Ganglion cells are shifted randomly by 12% of their separation to ac-\ncount for the non-ideal distribution on the hexagonal grid. Cones, bipolar and ganglion cells\nform the feed-forward path and horizontal and amacrine cells two lateral layers. Densities\nand receptive \ufb01eld sizes of photoreceptors and ganglion cells were adjusted to the anatom-\nical data available for the human retina at the different eccentricities studied (Fig.1B). The\nseparation of horizontal, bipolar and amacrine cells was scaled proportional to the cone\ndensity.\n\nEccentricity\n[deg]\n0\n5\n10\n15\n20\n\n1.00\n2.51\n2.98\n3.31\n3.52\n\nPSF scaling Vernier offset\n\n[arcsec]\n7\n46\n83\n92\n98\n\nTable 1: Spatial scaling of the PSF that simulates the optical blurring and of the vernier\noffset as a function of the eccentricity.\n\nThe photoreceptor model is a slightly modi\ufb01ed version of the mathematical description\ngiven by Hennig et al. [4]. It is originally based on a description by Schnapf et al. [8].\nThe voltage responses were tested against experimental data from the macaque monkey\nby Schneeweis and Schnapf [9]. To account for the sustained responses for strong, but\nbrief stimuli, the single initial activation stage [4] was replaced by three cascaded low-\npass \ufb01lters. This study focuses on human P On-center cells (or \u201cmidget\u201d cells). Receptive\n\ufb01eld sizes and densities were chosen according to anatomical data (Fig. 1). The center and\nsurround input of both cell types is weighted by overlapping Gaussian pro\ufb01les [10], where\nthe surround extends > 3.8 times the center input [11].\nOcular optical blurring has been accounted for by convolving the stimulus with the point-\nspread function (PSF) given by Westheimer et al. [12] for the fovea:\n\nP SF (\u03c1) = 0.933 \u00b7 e\u22122.59\u00b7\u03c11.36 + 0.047 \u00b7 e\u22122.34\u00b7\u03c11.74\n\n(1)\n\n\u03c1 is the radius in arcmin. For higher eccentricities two sets of simulations were performed,\none with a constant and one with a scaled PSF (Fig.1B). The \ufb01rst case is an approximation\nof the case when off-axis refractory errors of the ocular optics are corrected. Then alias-\ning occurs already at the level of the cone mosaic. The more realistic case corresponds\nto a scaled PSF because off-axis astigmatism and increasing cone aperture increase the\namount of blurring at higher eccentricities. Scaling factors were chosen to \ufb01t experimental\ndata (Tab. 1, [13]). Under these conditions, aliasing on the ganglion cell layer begins at\n5\u25e6 (Fig.1D).\nEye micromovements where modeled by shifting the retina randomly relative to the stim-\nulus by using a data \ufb01t by Eizenman et al. (Fig. 2A,B, [14]). They include the ocu-\nlar microtremor and fast and slow microsaccades (Fig. 2B). Two types of micromove-\nments were used in the simulations in this work: slow and fast microsaccades and the\nmicrotremor (MT) and only fast microsaccades and the tremor (FMT).\n\nA typical vernier stimulus has been used in the simulations. To remove the effect of the\nstimulus size, we used a bipartite \ufb01eld of 100% contrast with a small horizontal displace-\nment in the vertical half (Fig.1C). Simulations were carried out at \ufb01ve different retinal\neccentricities: in the fovea and at 5, 10, 15 and 20 deg. The vernier offset was scaled\nwith increasing eccentricity proportional to the ratio of the cone to ganglion cell separa-\ntion (Tab.1).\n\n\fFigure 2: Characteristics of the simulated eye-micromovements. A, Traces of the hori-\nzontal retinal displacement for the two tremor spectra used (top: MT, bottom: FMT, see\nMethods). B, Power spectra of the two cases from part A (dashed line: MT, dotted line:\nFMT) and the full spectrum given by Eigenman et al. (straight line, [14]). C, Responses\nof P-ganglion cells to a contrast step (100% contrast) without tremor (solid line) and with\neye micromovements (MT, dotted line). Horizontal alignment corresponds to the location\nof the cell relative to the stimulus (location of contrast step indicated by dotted line).\n\n3 Results\n\nFig. 2 summarizes the characteristics of simulated eye-micromovements.\nIn part A an\nexample for the horizontal displacement of the retina is shown for the two types of micro-\nmovements included in the model (MT and FMT, see Methods). Part B shows the corre-\nsponding power spectra. Fig. 2C shows the membrane potential of a simulated ganglion\ncell at different locations relative to a contrast step with and without eye micromovements.\nWhen the cell is located in the dark half of the contrast step, moving the light half of\nthe stimulus into its receptive \ufb01eld causes frequent strong depolarizations. For the reverse\ncase, when the dark half of the stimulus moves into the receptive \ufb01eld of a cell which was\npreviously excited, the membrane potential hyperpolarizes. These hyperpolarizations are\nweaker than the depolarizations in the former case because the photoreceptor response is\nasymmetric with respect to the to on- and offset of light. Light onset leads a to brief, strong\ntransient hyperpolarization whereas offset causes a slower response decay and a weaker\nphasic depolarization [4, 9].\n\nFig. 3A,E show the spatial response distribution on the ganglion cell layer 30ms after\nstimulus onset for two retinal eccentricities for the constant PSF. At 5\u25e6 eccentricity the\nvernier offset is well visible by eye by comparing the upper and lower half of the responses.\nAt 10\u25e6 however, upper and lower half look very similar, implying that vernier detection is\nnot possible.\n\nTo quantify the detectability of a vernier stimulus we performed a ROC analysis of the\nspatial response pro\ufb01les. This procedure is shown in Fig.3: First a horizontal cross-section\nof the spatial response pro\ufb01le on the ganglion cell layer is taken for the upper and lower\npart of the stimulus (B, F). The detectability of a vernier stimulus should be re\ufb02ected in the\npopulation average of the ganglion cell responses for upper and lower part of the stimulus.\nThis assumption re\ufb02ects the known convergence properties of the primary visual pathway,\nwhere each cortical cell receives input (via the LGN) from many ganglion cells. We used\n\n\fFigure 3: Spatial analysis of the vernier stimuli. A, Spatial response pro\ufb01les of the gan-\nglion cells to a vernier stimulus 30ms after stimulus onset (5\u25e6 retinal eccentricity, vernier\noffset 4500). The membrane potential is coded by gray levels. B, Spatial response pro\ufb01le\nfor the upper (black) and lower half (grey) of the responses in A (average over four rows).\nC, Spatial derivative of the curves in B, recti\ufb01ed at zero. D, ROC curve calculated from the\ncurves in C. Value of the integral of the ROC curve (shaded gray) is shown for each curve\n(detectability index). E-H The same analysis at 10\u25e6 retinal eccentricity and a vernier offset\nof 9200.\n\nan average of four rows of the ganglion cells for analysis. The resulting pro\ufb01les closely\n\ufb01t cumulative Difference of Gaussians functions, which is a consequence of the ganglion\ncell receptive \ufb01eld structure. In the next step, the spatial derivative of the response pro\ufb01le\nis calculated and recti\ufb01ed at the resting potential (C, G). This operation is similar to a\ncortical edge detection mechanism [15] and leads to Gaussian-like distributions. From\nthese curves it is possible to directly compute a ROC-curve (D, H). The integral of the\nROC curve, ranging from 0.5 to 1, is then taken as a direct measure of the detectability of\nthe vernier offset. This method combines the standard, model-free ROC-type analysis with\nbasic assumptions about the convergence properties in the primary visual pathway.\n\nEye-movements lead to temporal changes of the detectability. Thus, the integral of the\nROC curve, which we will call the \u201cdetectability index\u201d (DI), then varies over time. Fig.\n4A shows this effect for the \ufb01ve different retinal eccentricities studied and different types\nof micromovements using the scaled PSF. For each eccentricity, the stimulus has been\nplaced at \ufb01ve different locations relative to the ganglion cell receptive \ufb01elds. We found\nthat, without eye-micromovements and increasing eccentricities, the detectability strongly\ndepends on the location of the stimulus in the receptive \ufb01eld. This is not surprising when\none considers that spatial undersampling of the stimulus occurs at the ganglion cell layer.\nAt the fovea visual resolution is limited by the optics of the eye. At > 5\u25e6 eccentricity,\nthere are substantial \u201cgaps\u201d in the ganglion cell representation of the stimulus (see Fig.1B)\nwhich cause aliasing effects. Aliasing effects in the periphery due to undersampling has\nbeen reported in psychophysics [16].\n\nOcular micromovements leads to clearly visible effects (Fig. 4A). The noisy curves are\n\n\fFigure 4: Temporal analysis of the ROC curves. A, Detectability index as function\nof time at different retinal eccentricities and different stimulus displacements relative to\nthe ganglion cell positions (thick curves: resting eye, thin curves: slow+fast microsac-\ncades+tremor, grey curves: fast microsaccades+tremor). Stimulus offsets are shown above\nthe traces. B, Maximum of the curves in A at each eccentricity and location for the scaled\nPSF on a noisy ganglion cells grid. Only values are considered as a maximum where the\nDI stays above the mean for > 10ms. C, Maximal DI for the constant PSF.\n\nnow randomly oscillating across the smooth curves without micromovements. We note for\nmost curves obtained with tremor there is an interval of at least 10ms where the DI is\nsubstantially above its mean and equal or above the noise-free equivalent. Psychophysical\nevidence shows that detection tasks may require only short periods of as little as 5-10ms\nwhere the detectability must exceed threshold [17]. Thus in the retinal periphery the eye\nmicromovements have a bene\ufb01cial effect on the detectability by reducing aliasing.\n\nIn Fig. 4B, the maximum of DI at different stimulus locations is plotted as function of\nthe stimulus position. The maximum is de\ufb01ned as the largest value of the detectability\nindex within a > 10ms transient. The curves show the same effects as described above:\nPerformance remains the similar in the central and improves in the peripheral retina. If the\nmean value of DI instead of the maximum is considered, the effect is similar in the fovea,\nbut no performance increase can be observed in the periphery (not shown). Fig. 4C shows\nthe same analysis of responses for a constant PSF on a regular ganglion cell grid (see Fig.\n1B), where aliasing occurs already at the photoreceptor level. The effect is very similar to\nthat of the scaled PSF with stronger aliasing at higher eccentricities. However, at 10 and 15\ndeg, DI is lower for all cases because the disarray of the ganglion cells allows for improved\nspatial averaging.\n\nTo summarize the previous results, the mean value of each curve in Fig. 4B and C is cal-\nculated. This can be interpreted as the psychophysical performance of a subject after many\nstimulus repetitions. They are shown in Fig. 5A for the scaled and Fig. 5B for the constant\n\n\fFigure 5: Mean detectability index (DI) for the experiments in Fig.3A (left, constant PSF)\nand B (right, PSF scaled proportional to cone-ganglion cell convergence ratio) as function\nof the retinal eccentricity.\n\nPSF. The differences in DI at different eccentricities is a result of the stimulus scaling. For\nboth cases, eye micromovements increases the detectability at all eccentricities except in\nthe fovea. For the two types of eye micromovements, the maximal relative improvement\nof DI happens at different eccentricities. The \ufb01rst type, comprising microsaccades and\ntremor, frequently shifts the stimulus across adjoining ganglion cells at eccentricities 20\u25e6.\nThe second type has a smaller amplitude, thus the excitation of nearby ganglion cells is\nmost ef\ufb01cient at 10\u25e6. Thus, the effect depends on the spatial extend of the eye movements.\nAt 20\u25e6, DI is much lower for the scaled PSF on a noisy ganglion cell grid than for the\nconstant PSF on the regular grid. Because DI is consistently lower in the latter case for\nthe other eccentricities, this indicates that here the effect of the spatial disarray can not be\ncountered by spatial averaging of just four rows of ganglion cells.\n\nTaken together, the results from the simulations shown here imply that a complex interplay\nof different factors affect the detectability of hyperacuity stimuli. Indeed the quantitative\nresults from the model are very sensitive to changes of certain parameters (e.g. cell density).\nEqually, a great variability in human psychophysical performance exists. However, the\neffect of eye micromovements is consistent across the two cases shown here.\n\n4 Discussion\n\nOur results suggest that eye-micromovements contribute to visual hyperacuity in the pe-\nripheral visual \ufb01eld. By simulating ganglion cell responses for vernier stimuli using a\nrealistic model and applying model-free ideal observer analysis, we show that in the retinal\nperiphery eye-micromovements reduce the effect of aliasing due to neural undersampling.\nThis leads to a higher detectability of hyperacuity stimuli. There has been a successful\nattempt to use small, continuous \u201cscanning\u201d movements to increase the resolution of a\nlow resolution sensor array as a technical application [18]. We show that this principle\ncan indeed be employed by vertebrates to improve visual acuity in certain (hyperacuity)\ntasks. However, eye movements have the reverse effect on detection tasks that require\naliasing. Packer and Williams [19] have shown that in a high frequency (aliasing) grating\ndetection task contrast thresholds are low for very brief and long presentation durations.\nFor intermediate presentation times the threshold increases substantially. Because detec-\ntion relies on aliasing, it requires a resting eye. This is more likely for very brief and\nlong presentation times. For intermediate intervals, motion prevents aliasing. In hyperacu-\nity, eye-micromovements increase detectability and we expect an asymptotic decrease of\nthresholds as function of the presentation time.\n\nThe question arises how eye-micromovements affect human psychophysical performance.\n\n\fWe predict an in\ufb02uence of the effect of stimulus presentation time for vernier targets be-\ntween the central and peripheral retina. We would also expect an increase of detection\nthresholds under stabilized eye conditions in the periphey. This and further experiments\nalso suggest that eye micromovements generally in\ufb02uence detection tasks that are per-\nformed close to the psychophysical threshold. It is further possible to directly apply the\nexperimental procedure that was used in this work in an electrophysiological study. Specif-\nically, it is possible to record from one ganglion cell with many different stimulus locations.\nThese responses can then be used to reconstruct a spatial response pro\ufb01le equivalent to our\nsimulated activity distribution (Fig.3B, F) and ROC analysis can be applied.\n\nReferences\n[1] H.L. Averill and F.W. Weymouth. Visual perception and the retinal mosaic. II. The in\ufb02uence of\n\neye-movements on the displacement threshold. J Comp Psychol, 5:147\u2013176, 1925.\n\n[2] W.H. Marshall and S.A. Talbot. Recent evidence for neural mechanisms in vision leading to a\n\ngeneral theory of sensory acuity. Biol Symp, 7:117\u2013164, 1942.\n\n[3] R.M. Steinman and J.Z. Levinson. Eye movements and their role in visual and cognitive pro-\ncesses, chapter The role of eye movement in the detection of contrast and spatial detail, pages\n115\u2013212. Elsevier Science, 1990.\n\n[4] M.H. Hennig, K. Funke, and F. W\u00a8org\u00a8otter. The in\ufb02uence of different retinal subcircuits on the\n\nnonlinearity of ganglion cell behavior. J Neurosci, 22:8726\u20138738, 2002.\n\n[5] J. Sj\u00a8ostrand, V. Olsson, Z. Popovic, and N. Conradi. Quantitative estimations of foveal and\n\nextra-foveal retinal circuitry in humans. Vision Res, 39:2987\u20132998, 1999.\n\n[6] A.K. Goodchild, K.K. Ghosh, and P.R. Martin. Comparison of photoreceptor spatial density\nand ganglion cell morphology in the retina of human, macaque monkey, cat, and the marmoset\ncallithrix jacchus. J Comp Neurol, 366:55\u201375, 1996.\n\n[7] D.M. Dacey and M.R. Petersen. Dendritic \ufb01eld size and morphology of midget and parasol\n\nganglion cells in the human retina. Proc Natl Acad Sci USA, 89:9666\u20139670, 1992.\n\n[8] J.L. Schnapf, B.J. Nunn, M. Meister, and D.A. Baylor. Visual transduction in cones of the\n\nmonkey macaca fascicularis. J Physiol, 427:681\u2013713, 1990.\n\n[9] D.M. Schneeweis and J.L. Schnapf. The photovoltage of marcaque cone photoreceptors: adap-\n\nation, noise and kinetics. J Neurosci, 19(4):1203\u20131216, 1999.\n\n[10] R.W. Rodieck and J. Stone. Analysis of receptive \ufb01elds of cat retinal ganglion cells. J Neuro-\n\nphysiol, 28:833\u2013849, 1965.\n\n[11] L.J. Croner and E. Kaplan. Receptive \ufb01elds of P and M ganglion cells across the primate retina.\n\nVision Res, 35(1):7\u201324, 1995.\n\n[12] G. Westheimer. Handbook of Perception and Human Performance, volume 1, chapter The eye\n\nas an optical instrument. John Wiley & Sons, New York, 1986.\n\n[13] L.N. Thibos, D.L. Still, and Bradley A. Characterization of spatial aliasing and contrast sensi-\n\ntivity in peripheral vision. Vision Res, 36:249\u201358, 1996.\n\n[14] M. Eizenman, P.E. Hallett, and R.C. Frecker. Power spectra for ocular drift and tremor. Vision\n\nRes, 25:1635\u20131640, 1985.\n\n[15] D.H. Hubel and T.N. Wiesel. Receptive \ufb01elds, binocular interaction, and functional architecture\n\nin the cat\u2019s visual cortex. J Physiol, 160:106\u2013154, 1962.\n\n[16] L.N. Thibos, D.J. Walsh, and Cheney F.E. Vision beyond the resolution limit: aliasing in the\n\nperiphery. Vision Res, 27:2193\u20132197, 1987.\n\n[17] A.B. Watson. Handbook of perception and human performance, volume 1, chapter Temporal\n\nsensitivity. Wiley, New York, 1986.\n\n[18] Landolt O. and Mitros A. Visual sensor with resolution enhancement by mechanical vibrations.\n\nAutonomous Robots, 11:233\u2013239, 2001.\n\n[19] O. Packer and D.R. Williams. Blurring by \ufb01xational eye movements. Vision Res, 32:1931\u20131939,\n\n1992.\n\n\f", "award": [], "sourceid": 2514, "authors": [{"given_name": "Matthias", "family_name": "Hennig", "institution": null}, {"given_name": "Florentin", "family_name": "W\u00f6rg\u00f6tter", "institution": null}]}