{"title": "Multiscale Semi-Markov Dynamics for Intracortical Brain-Computer Interfaces", "book": "Advances in Neural Information Processing Systems", "page_first": 868, "page_last": 878, "abstract": "Intracortical brain-computer interfaces (iBCIs) have allowed people with tetraplegia to control a computer cursor by imagining the movement of their paralyzed arm or hand. State-of-the-art decoders deployed in human iBCIs are derived from a Kalman filter that assumes Markov dynamics on the angle of intended movement, and a unimodal dependence on intended angle for each channel of neural activity. Due to errors made in the decoding of noisy neural data, as a user attempts to move the cursor to a goal, the angle between cursor and goal positions may change rapidly. We propose a dynamic Bayesian network that includes the on-screen goal position as part of its latent state, and thus allows the person\u2019s intended angle of movement to be aggregated over a much longer history of neural activity. This multiscale model explicitly captures the relationship between instantaneous angles of motion and long-term goals, and incorporates semi-Markov dynamics for motion trajectories. We also introduce a multimodal likelihood model for recordings of neural populations which can be rapidly calibrated for clinical applications. In offline experiments with recorded neural data, we demonstrate significantly improved prediction of motion directions compared to the Kalman filter. We derive an efficient online inference algorithm, enabling a clinical trial participant with tetraplegia to control a computer cursor with neural activity in real time. The observed kinematics of cursor movement are objectively straighter and smoother than prior iBCI decoding models without loss of responsiveness.", "full_text": "Multiscale Semi-Markov Dynamics for\nIntracortical Brain-Computer Interfaces\n\nDaniel J. Milstein \u2217\n\ndaniel_milstein@alumni.brown.edu\n\nJason L. Pacheco \u2020\npachecoj@mit.edu\n\nLeigh R. Hochberg \u2021 \u00a7 \u00b6\nleigh_hochberg@brown.edu\n\nJohn D. Simeral \u2021 \u00a7\n\njohn_simeral@brown.edu\n\nBeata Jarosiewicz (cid:107) \u00a7 \u2217\u2217\nbeataj@stanford.edu\n\nErik B. Sudderth \u2020\u2020 \u2217\nsudderth@uci.edu\n\nAbstract\n\nIntracortical brain-computer interfaces (iBCIs) have allowed people with tetraplegia\nto control a computer cursor by imagining the movement of their paralyzed arm\nor hand. State-of-the-art decoders deployed in human iBCIs are derived from a\nKalman \ufb01lter that assumes Markov dynamics on the angle of intended movement,\nand a unimodal dependence on intended angle for each channel of neural activity.\nDue to errors made in the decoding of noisy neural data, as a user attempts to\nmove the cursor to a goal, the angle between cursor and goal positions may change\nrapidly. We propose a dynamic Bayesian network that includes the on-screen goal\nposition as part of its latent state, and thus allows the person\u2019s intended angle of\nmovement to be aggregated over a much longer history of neural activity. This\nmultiscale model explicitly captures the relationship between instantaneous angles\nof motion and long-term goals, and incorporates semi-Markov dynamics for motion\ntrajectories. We also introduce a multimodal likelihood model for recordings\nof neural populations which can be rapidly calibrated for clinical applications.\nIn of\ufb02ine experiments with recorded neural data, we demonstrate signi\ufb01cantly\nimproved prediction of motion directions compared to the Kalman \ufb01lter. We derive\nan ef\ufb01cient online inference algorithm, enabling a clinical trial participant with\ntetraplegia to control a computer cursor with neural activity in real time. The\nobserved kinematics of cursor movement are objectively straighter and smoother\nthan prior iBCI decoding models without loss of responsiveness.\n\nIntroduction\n\n1\nParalysis of all four limbs from injury or disease, or tetraplegia, can severely limit function, inde-\npendence, and even sometimes communication. Despite its inability to effect movement in muscles,\nneural activity in motor cortex still modulates according to people\u2019s intentions to move their paralyzed\narm or hand, even years after injury [Hochberg et al., 2006, Simeral et al., 2011, Hochberg et al.,\n\nsachusetts General Hospital, Boston, MA, USA.\n\n\u2217Department of Computer Science, Brown University, Providence, RI, USA.\n\u2020Computer Science and Arti\ufb01cial Intelligence Laboratory, MIT, Cambridge, MA, USA.\n\u2021School of Engineering, Brown University, Providence, RI, USA; and Department of Neurology, Mas-\n\u00a7Rehabilitation R&D Service, Department of Veterans Affairs Medical Center, Providence, RI, USA; and\nBrown Institute for Brain Science, Brown University, Providence, RI, USA.\n\u00b6Department of Neurology, Harvard Medical School, Boston, MA, USA.\n(cid:107)Department of Neuroscience, Brown University, Providence, RI, USA.\n\u2217\u2217Present af\ufb01liation: Dept. of Neurosurgery, Stanford University, Stanford, CA, USA.\n\u2020\u2020Department of Computer Science, University of California, Irvine, CA, USA.\n\n31st Conference on Neural Information Processing Systems (NIPS 2017), Long Beach, CA, USA.\n\n\fFigure 1: A microelectrode array (left) is implanted in the motor cortex (center) to record electrical activity.\nVia this activity, a clinical trial participant (right, lying on his side in bed) then controls a computer cursor with\nan iBCI. A cable connected to the electrode array via a transcutaneous connector (gray box) sends neural signals\nto the computer for decoding. Center drawing from Donoghue et al. [2011] and used with permission of the\nauthor. The right image is a screenshot of a video included in the supplemental material that demonstrates real\ntime decoding via our MSSM model.\n\n2012, Collinger et al., 2013]. Intracortical brain-computer interfaces (iBCIs) utilize neural signals\nrecorded from implanted electrode arrays to extract information about movement intentions. They\nhave enabled individuals with tetraplegia to control a computer cursor to engage in tasks such as\non-screen typing [Bacher et al., 2015, Jarosiewicz et al., 2015, Pandarinath et al., 2017], and to regain\nvolitional control of their own limbs [Ajiboye et al., 2017].\nCurrent iBCIs are based on a Kalman \ufb01lter that assumes the vector of desired cursor movement\nevolves according to Gaussian random walk dynamics, and that neural activity is a Gaussian-corrupted\nlinear function of this state [Kim et al., 2008]. In Sec. 2, we review how the Kalman \ufb01lter is applied\nto neural decoding, and studies of the motor cortex by Georgopoulos et al. [1982] that justify its use.\nIn Sec. 3, we improve upon the Kalman \ufb01lter\u2019s linear observation model by introducing a \ufb02exible,\nmultimodal likelihood inspired by more recent research [Amirikian and Georgopulos, 2000]. Sec. 4\nthen proposes a graphical model (a dynamic Bayesian network [Murphy, 2002]) for the relationship\nbetween the angle of intended movement and the intended on-screen goal position. We derive an\nef\ufb01cient inference algorithm via an online variant of the junction tree algorithm [Boyen and Koller,\n1998]. In Sec. 5, we use recorded neural data to validate the components of our multiscale semi-\nMarkov (MSSM) model, and demonstrate signi\ufb01cantly improved prediction of motion directions in\nof\ufb02ine analysis. Via a real time implementation of the inference algorithm on a constrained embedded\nsystem, we then evaluate online decoding performance as a participant in the BrainGate21 iBCI pilot\nclinical trial uses the MSSM model to control a computer cursor with his neural activity.\n\n2 Neural decoding via a Kalman \ufb01lter\n\nThe Kalman \ufb01lter is the current state-of-the-art for iBCI decoding. There are several con\ufb01gurations\nof the Kalman \ufb01lter used to enable cursor control in contemporary iBCI systems [Pandarinath et al.,\n2017, Jarosiewicz et al., 2015, Gilja et al., 2015] and there is no broad consensus in the iBCI \ufb01eld on\nwhich is most suited for clinical use. In this paper, we focus on the variant described by Jarosiewicz\net al. [2015].\nParticipants in the BrainGate2 clinical trial receive one or two microelectrode array implants in the\nmotor cortex (see Fig. 1). The electrical signals recorded by this electrode array are then transformed\n(via signal processing methods designed to reduce noise) into a D-dimensional neural activity vector\nzt \u2208 RD, sampled at 50 Hz. From the sequence of neural activity, the Kalman \ufb01lter estimates the\nlatent state xt \u2208 R2, a vector pointing in the intended direction of cursor motion. The Kalman \ufb01lter\nassumes a jointly Gaussian model for cursor dynamics and neural activity,\n\nxt | xt\u22121 \u223c N (Axt\u22121, W ),\n\n(1)\nwith cursor dynamics A \u2208 R2\u00d72, process noise covariance W \u2208 R2\u00d72, and (typically non-diagonal)\nobservation covariance Q \u2208 RD\u00d7D. At each time step, the on-screen cursor\u2019s position is moved by\nthe estimated latent state vector (decoder output) scaled by a constant, the speed gain.\nThe function relating neural activity to some measurable quantity of interest is called a tuning curve.\nA common model of neural activity in the motor cortex assumes that each neuron\u2019s activity is highest\n\nzt | xt \u223c N (b + Hxt, Q),\n\n1Caution: Investigational Device. Limited by Federal Law to Investigational Use.\n\n2\n\n\ffor some preferred direction of motion, and lowest in the opposite direction, with intermediate activity\noften resembling a cosine function. This cosine tuning model is based on pioneering studies of\nthe motor cortex of non-human primates [Georgopoulos et al., 1982], and is commonly used (or\nimplicitly assumed) in iBCI systems because of its mathematical simplicity and tractability.\nExpressing the inner product between vectors via the cosine of the angle between them, the expected\nneural activity of the jth component of Eq. (1) can be written as\nj xt = bj + ||xt|| \u00b7 ||hj|| \u00b7 cos\n\nE[ztj | xt] = bj + hT\n\n\u03b8t \u2212 atan\n\n(cid:18)\n\n(2)\n\n(cid:18) hj2\n\n(cid:19)(cid:19)\n\nhj1\n\n,\n\n1 , . . . , hT\n\nwhere \u03b8t is the intended angle of movement at timestep t, bj is the baseline activity rate for channel j,\nand hj is the jth row of the observation matrix H = (hT\nD)T . If xt is further assumed\nto be a unit vector (a constraint not enforced by the Kalman \ufb01lter), Eq. (2) simpli\ufb01es to hT\nj xt =\nmj cos(\u03b8t \u2212 pj), where mj is the modulation of the tuning curve and pj speci\ufb01es the angular location\nof the peak of the cosine tuning curve (the preferred direction). Thus, cosine tuning models are linear.\nTo collect labeled training data for decoder calibration, the participant is asked to attempt to move\na cursor to prompted target locations. We emphasize that although the clinical random target\ntask displays only one target at a time, this target position is unknown to the decoder. Labels are\nconstructed for the neural activity patterns by assuming that at each 20ms time step, the participant\nintends to move the cursor straight to the target [Jarosiewicz et al., 2015, Gilja et al., 2015]. These\nlabeled data are used to \ufb01t the observation matrix H and neuron baseline rates (biases) b via ridge\nregression. The observation noise covariance Q is estimated as the empirical covariance of the\nresiduals. The state dynamics matrix A and process covariance matrix W may be tuned to adjust the\nresponsiveness of the iBCI system.\n\n3 Flexible tuning likelihoods\n\nThe cosine tuning model reviewed in the previous section has several shortcomings. First, motor\ncortical neurons that have unimodal tuning curves often have narrower peaks that are better described\nby von Mises distributions [Amirikian and Georgopulos, 2000]. Second, tuning can be multimodal.\nThird, neural features used for iBCI decoding may capture the pooled activity of several neurons,\nnot just one [Fraser et al., 2009]. While bimodal von Mises models were introduced by Amirikian\nand Georgopulos [2000], up to now iBCI decoders based on von Mises tuning curves have only\nemployed unimodal mean functions proportional to a single von Mises density [Koyama et al., 2010].\nIn contrast, we introduce a multimodal likelihood proportional to an arbitrary number of regularly\nspaced von Mises densities and incorporate this likelihood into an iBCI decoder. Moreover, we can\nef\ufb01ciently \ufb01t parameters of this new likelihood via ridge regression. Computational ef\ufb01ciency is\ncrucial to allow rapid calibration in clinical applications.\nLet \u03b8t \u2208 [0, 2\u03c0) denote the intended angle of cursor movement at time t. The \ufb02exible tuning likelihood\ncaptures more complex neural activity distributions via a regression model with nonlinear features:\n\nzt | \u03b8t \u223c N(cid:0)b + wT \u03c6(\u03b8t), Q(cid:1) ,\n\n\u03c6k(\u03b8t) = exp [\u0001 cos (\u03b8t \u2212 \u03d5k)] .\n\n(3)\n\nThe features are a set of K von Mises basis functions \u03c6(\u03b8) = (\u03c61(\u03b8), . . . , \u03c6K(\u03b8))T . Basis functions\n\u03c6k(x) are centered on a regular grid of angles \u03d5k, and have tunable concentration \u0001.\nUsing human neural data recorded during cued target tasks, we compare regression \ufb01ts for the \ufb02exible\ntuning model to the standard cosine tuning model (Fig. 2). In addition to providing better \ufb01ts for\nchannels with complex or multimodal activity, the \ufb02exible tuning model also provides good \ufb01ts to\napparently cosine-tuned signals. This leads to higher predictive likelihoods for held-out data, and as\nwe demonstrate in Sec. 5, more accurate neural decoding algorithms.\n\n4 Multiscale Semi-Markov Dynamical Models\n\nThe key observation underlying our multiscale dynamical model is that the sampling rate used for\nneural decoding (typically around 50 Hz) is much faster than the rate that the goal position changes\n(under normal conditions, every few seconds). In addition, frequent but small adjustments of cursor\naim angle are required to maintain a steady heading. State-of-the-art Kalman \ufb01lter approaches to iBCIs\n\n3\n\n\fFigure 2: Flexible tuning curves. Each panel shows the empirical mean and standard deviation (red) of\nexample neural signals recorded from a single intracortical electrode while a participant is moving within\n45 degrees of a given direction in a cued target task. These signals can violate the assumptions of a cosine\ntuning model (black), as evident in the left two examples. The \ufb02exible regression likelihood (cyan) captures\nneural activity with varying concentration (left) and multiple tuning directions (center), as well as cosine-tuned\nsignals (right). Because neural activity from individual electrodes is very noisy (the standard deviation within\neach angular bin exceeds the change in mean activity across angles), information from multiple electrodes is\naggregated over time for effective decoding.\n\nare incapable of capturing these multiscale dynamics since they assume \ufb01rst-order Markov dependence\nacross time and do not explicitly represent goal position. To cope with this, hyperparameters of the\nlinear Gaussian dynamics must be tuned to simultaneously remain sensitive to frequent directional\nadjustments, but not so sensitive that cursor dynamics are dominated by transient neural activity.\nOur proposed MSSM decoder, by contrast, explicitly represents goal position in addition to cursor\naim angle. Through the use of semi-Markov dynamics, the MSSM enables goal position to evolve\nat a different rate than cursor angle while allowing for a high rate of neural data acquisition. In this\nway, the MSSM can integrate across different timescales to more robustly infer the (unknown) goal\nposition and the (also unknown) cursor aim. We introduce the model in Sec. 4.1 and 4.2. We derive\nan ef\ufb01cient decoding algorithm, based on an online variant of the junction tree algorithm, in Sec. 4.3.\n\n4.1 Modeling Goals and Motion via a Dynamic Bayesian Network\n\nThe MSSM directed graphical model (Fig. 3) uses a structured latent state representation, sometimes\nreferred to as a dynamic Bayesian network [Murphy, 2002]. This factorization allows us to discretize\nlatent state variables, and thereby support non-Gaussian dynamics and data likelihoods. At each time\nt we represent discrete cursor aim \u03b8t as 72 values in [0, 2\u03c0) and goal position gt as a regular grid\nof 40 \u00d7 40 = 1600 locations (see Fig. 4). Each cell of the grid is small compared to elements of a\ngraphical interface. Cursor aim dynamics are conditioned on goal position and evolve according to a\nsmoothed von Mises distribution:\n\nvMS(\u03b8t | gt, pt) (cid:44) \u03b1/2\u03c0 + (1 \u2212 \u03b1)vonMises(\u03b8t | a(gt, pt), \u00af\u03ba).\n\n(4)\nHere, a(g, p) = tan\u22121((gy \u2212 py)/(gx \u2212 px)) is the angle from the cursor p = (px, py) to the goal\ng = (gx, gy), and the concentration parameter \u00af\u03ba encodes the expected accuracy of user aim. Neural\nactivity from some participants has short bursts of noise during which the learned angle likelihood is\ninaccurate; the outlier weight 0 < \u03b1 < 1 adds robustness to these noise bursts.\n\n4.2 Multiscale Semi-Markov Dynamics\n\nThe \ufb01rst-order Markov assumption made by existing iBCI decoders (see Eq. (1)) imposes a geometric\ndecay in state correlation over time. For example, consider a scalar Gaussian state-space model:\nxt = \u03b2xt\u22121 + v, v \u223c N (0, \u03c32). For time lag k > 0, the covariance between two states cov(xt, xt+k)\ndecays as \u03b2\u2212k. This weak temporal dependence is highly problematic in the iBCI setting due to the\nmismatch between downsampled sensor acquisition rates used for decoding (typically around 50Hz,\nor 20ms per timestep) and the time scale at which the desired goal position changes (seconds).\nWe relax the \ufb01rst-order Markov assumption via a semi-Markov model of state dynamics [Yu, 2010].\nSemi-Markov models, introduced by Levy [1954] and Smith [1955], divide the state evolution into\ncontiguous segments. A segment is a contiguous series of timesteps during which a latent variable is\nunchanged. The conditional distribution over the state at time xt depends not only on the previous\nstate xt\u22121, but also on a duration dt which encodes how long the state is to remain unchanged:\n\n4\n\nAngle (degrees)-180-135-90-4504590135180Spike power measurement5.566.577.58DataCosine fitFlexible fitAngle (degrees)-180-135-90-4504590135180Spike power measurement99.51010.51111.51212.513DataCosine fitFlexible fitAngle (degrees)-180-135-90-4504590135180Spike power measurement5.45.455.55.555.65.655.75.755.8DataCosine fitFlexible fit\fMultiscale Semi-Markov Model\n\nJunction Tree for Online Decoding\n\nFigure 3: Multiscale semi-Markov dynamical model. Left: The multiscale directed graphical model of how\ngoal positions gt, angles of aim \u03b8t, and observed cursor positions pt evolve over three time steps. Dashed nodes\nare counter variables enabling semi-Markov dynamics. Right: Illustration of the junction tree used to compute\nmarginals for online decoding, as in Boyen and Koller [1998]. Dashed edges indicate cliques whose potentials\ndepend on the marginal approximations at time t \u2212 1. The inference uses an auxiliary variable rt (cid:44) a(gt, pt),\nthe angle from the cursor to the current goal, to reduce computation and allow inference to operate in real time.\n\np(xt | xt\u22121, dt). Duration is modeled via a latent counter variable, which is drawn at the start of\neach segment and decremented deterministically until it reaches zero, at which point it is resampled.\nIn this way the semi-Markov model is capable of integrating information over longer time horizons,\nand thus less susceptible to intermittent bursts of sensor noise.\nWe de\ufb01ne separate semi-Markov dynamical models for the goal position and the angle of intended\nmovement. As detailed in the supplement, in experiments our duration distributions were uniform,\nwith parameters informed by knowledge about typical trajectory durations and reaction times.\nGoal Dynamics A counter ct encodes the temporal evolution of the semi-Markov dynamics on\ngoal positions: ct is drawn from a discrete distribution p(c) at the start of each trajectory, and then\ndecremented deterministically until it reaches zero. (During decoding we do not know the value of\nthe counter, and maintain a posterior probability distribution over its value.) The goal position gt\nremains unchanged until the goal counter reaches zero, at which point with probability \u03b7 we resample\na new goal, and we keep the same goal with the remaining probability 1 \u2212 \u03b7:\nct = ct\u22121 \u2212 1, ct\u22121 > 0, Decrement\nct\u22121 = 0,\n\np(ct | ct\u22121) =\n\n(5)\n\nSample new counter\nOtherwise\n\np(ct),\n0,\n\np(gt | ct\u22121, gt\u22121) =\n\nG + (1 \u2212 \u03b7),\n\u03b7 1\n\u03b7 1\nG ,\n0,\n\nct\u22121 > 0, gt = gt\u22121, Goal position unchanged\nct\u22121 = 0, gt = gt\u22121, Sample same goal position\nct\u22121 = 0, gt (cid:54)= gt\u22121, Sample new goal position\n\nOtherwise\n\n(6)\n\nCursor Angle Dynamics We de\ufb01ne similar semi-Markov dynamics for the cursor angle via an aim\ncounter bt. Once the counter reaches zero, we sample a new aim counter value from the discrete\ndistribution p(b), and a new cursor aim angle from the smoothed von Mises distribution of Eq. (4):\n\np(bt | bt\u22121) =\n\np(bt),\n0,\np(\u03b8t | bt\u22121, \u03b8t\u22121, pt, gt) =\n\n(cid:26) \u03b8t\u22121\n\nbt = bt\u22121 \u2212 1, bt\u22121 > 0, Decrement\nbt\u22121 = 0,\n\nSample new counter\nOtherwise\n\nvMS(\u03b8t | gt, pt)\n\nbt\u22121 > 0, Keep cursor aim\nbt\u22121 = 0, Sample new cursor aim\n\n(cid:40) 1,\n\uf8f1\uf8f4\uf8f2\uf8f4\uf8f3 1,\n(cid:40) 1,\n\n4.3 Decoding via Approximate Online Inference\n\nEf\ufb01cient decoding is possible via an approximate variant of the junction tree algorithm [Boyen and\nKoller, 1998]. We approximate the full posterior at time t via a partially factorized posterior:\n\np(gt, ct, \u03b8t, bt | z1...t) \u2248 p(gt, ct | z1...t)p(\u03b8t, bt | z1...t).\n\n5\n\n(7)\n\n(8)\n\n(9)\n\nGoal reconsideration counterObservationCursor positionAngle of aimGoal positionAim adjustmentcounter\fGoal Positions\n\n0s\n\n0.5s\n\n1s\n\nFigure 4: Decoding goal positions. The MSSM represents goal position via a regular grid of 40\u00d7 40 locations\n(upper left). For one real sequence of recorded neural data, the above panels illustrate the motion of the cursor\n(white dot) to the user\u2019s target (red circle). Panels show the marginal posterior distribution over goal positions at\n0.5s intervals (25 discrete time steps of graphical model inference). Yellow goal states have highest probability,\ndark blue goal states have near-zero probability. Note the temporal aggregation of directional cues.\n\n1.5s\n\n2s\n\nHere p(gt, ct | z1...t) is the marginal on the goal position and goal counter, and p(\u03b8t, bt | z1...t) is the\nmarginal on the angle of aim and the aim counter. Note that in this setting goal position gt and cursor\naim \u03b8t, as well as their respective counters ct and bt, are unknown and must be inferred from neural\ndata. At each inference step we use the junction tree algorithm to compute state marginals at time t,\nconditioned on the factorized posterior approximation from time step t \u2212 1 (see Fig. 3). Boyen and\nKoller [1998] show that this technique has bounded approximation error over time, and Murphy and\nWeiss [2001] show this as a special case of loopy belief propagation.\nDetailed inference equations are derived in the supplemental material. Given G goal positions\nand A discrete angle states, each temporal update for our online decoder requires O(GA + A2)\noperations. In contrast, the exact junction tree algorithm would require O(G2A2) operations; for\npractical numbers of goals G, realtime implementation of this exact decoder is infeasible.\nFigure 4 shows several snapshots of the marginal posterior over goal position. At each time the\n, computed by taking an average of\nthe directions needed to get to each possible goal, weighted by the inferred probability that each goal\nis the participant\u2019s true target. This vector is smaller in magnitude when the decoder is less certain\nabout the direction in which the intended goal lies, which has the practical bene\ufb01t of allowing the\nparticipant to slow down near the goal.\n\nMSSM decoder moves the cursor along the vector E(cid:104) gt\u2212pt\n\n(cid:105)\n\n(cid:107)gt\u2212pt(cid:107)\n\n5 Experiments\n\nWe evaluate all decoders under a variety of conditions and a range of con\ufb01gurations for each decoder.\nControlled of\ufb02ine evaluations allow us to assess the impact of each proposed innovation. To analyze\nthe effects of our proposed likelihood and multiscale dynamics in isolation, we construct a baseline\nhidden Markov model (HMM) decoder using the same discrete representation of angles as the\nMSSM, and either cosine-tuned or \ufb02exible likelihoods. Our \ufb01ndings show that the of\ufb02ine decoding\nperformance of the MSSM is superior in all respects to baseline models.\nWe also evaluate the MSSM decoder in two online clinical research sessions, and compare head-\nto-head performance with the Kalman \ufb01lter. Previous studies have tested the Kalman \ufb01lter under a\nvariety of responsive parameter con\ufb01gurations and found a tradeoff between slow, smooth control\nversus fast, meandering control [Willett et al., 2016, 2017]. Through comparisons to the Kalman, we\ndemonstrate that the MSSM decoder maintains smoother and more accurate control at comparable\nspeeds. These realtime results are preliminary since we have yet to evaluate the MSSM decoder on\nother clinical metrics such as communication rate.\n\n6\n\n\fFigure 5: Of\ufb02ine decoding. Mean squared error of angular prediction for a variety of decoders, where each\ndecoder processes the same sets of recorded data. We analyze 24 minutes (eight 3-minute blocks) of neural data\nrecorded from participant T9 on trial days 546 and 552. We use one block for testing and the remainder for\ntraining, and average errors across the choice of test block. On the left, we report errors over all time points. On\nthe right, we report errors on time points during which the cursor was outside a \ufb01xed distance from the target.\nFor both analyses, we exclude the initial 1s after target acquisition, during which the ground truth is unreliable.\nTo isolate preprocessing effects, the plots separately report the Kalman without preprocessing (\u201craw\u201d). Dynamics\neffects are isolated by separately evaluating HMM dynamics (\u201cHMM\u201d), and likelihood effects are isolated by\nseparately evaluating \ufb02exible likelihood and cosine tuning in each con\ufb01guration. \u201cKalmanBC\u201d denotes the\nKalman \ufb01lter with an additional kinematic bias-correction heuristic [Jarosiewicz et al., 2015].\n\n5.1 Of\ufb02ine evaluation\n\nWe perform of\ufb02ine analysis using previously recorded data from two historical sessions of iBCI use\nwith a single participant (T9). During each session the participant is asked to perform a cued target\ntask in which a target appears at a random location on the screen and the participant attempts to move\nthe cursor to the target. Once the target is acquired or after a timeout (10 seconds), a new target is\npresented at a different location. Each session is composed of several 3 minute segments or blocks.\nTo evaluate the effect of each innovation we compare to an HMM decoder. This HMM baseline\nisolates the effect of our \ufb02exible likelihood since, like the Kalman \ufb01lter, it does not model goal\npositions and assumes \ufb01rst-order Markov dynamics. Let \u03b8t be the latent angle state at time t and\nx(\u03b8) = (cos(\u03b8), sin(\u03b8))T the corresponding unit vector. We implement a pair of HMM decoders for\ncosine tuning and our proposed \ufb02exible tuning curves,\n\nzt | \u03b8t \u223c N(cid:0)b + wT \u03c6(\u03b8t), Q(cid:1)\n(cid:125)\n(cid:124)\n\n(cid:123)(cid:122)\n\n(cid:125)\n\nCosine HMM\n\nFlexible HMM\n\nzt | \u03b8t \u223c N (b + Hx(\u03b8t), Q)\n,\n\n(cid:124)\n\n(cid:123)(cid:122)\n\nHere, \u03c6(\u00b7) are the basis vectors de\ufb01ned in Eq. (3). The state \u03b8t is discrete, taking one of 72 angular\nvalues equally spaced in [0, 2\u03c0), the same discretization used by the MSSM. Continuous densities\nare appropriately normalized. Unlike the linear Gaussian state-space model, the HMMs constrain\nlatent states to be valid angles (equivalently, unit vectors) rather than arbitrary vectors in R2.\nWe analyze decoder accuracy within each session using a leave-one-out approach. Speci\ufb01cally, we\ntest the decoder on each held-out block using the remaining blocks in the same session for training.\nWe report MSE of the predicted cursor direction, using the unit vector from the cursor to the target\nas ground truth, and normalizing decoder output vectors. We used the same recorded data for each\ndecoder. See the supplement for further details.\nFigure 5 summarizes the \ufb01ndings of the of\ufb02ine comparisons for a variety of decoder con\ufb01gurations.\nFirst, we evaluate the effect of preprocessing the data by taking the square root, applying a low-pass\nIIR \ufb01lter, and clipping the data outside a 5\u03c3 threshold, where \u03c3 is the empirical standard deviation of\ntraining data. This preprocessing signi\ufb01cantly improves accuracy for all decoders. The MSSM model\ncompares favorably to all con\ufb01gurations of the Kalman decoders. The majority of bene\ufb01t comes from\nthe semi-Markov dynamical model, but additional gains are observed when including the \ufb02exible\ntuning likelihood. Finally, it has been observed that the Kalman decoder is sensitive to outliers for\nwhich Jarosiewicz et al. [2015] propose a correction to avoid biased estimates. We test the Kalman\n\ufb01lter with and without this correction.\n\n7\n\nKalman (raw)KalmanCosine HMMFlexible HMMKalmanBC (raw)KalmanBCCosine MSSMFlexible MSSM00.050.10.150.20.25Mean squared errorKalman (raw)KalmanCosine HMMFlexible HMMKalmanBC (raw)KalmanBCCosine MSSMFlexible MSSM00.050.10.150.20.25Mean squared error\fFigure 6: Realtime decoding. A realtime comparison of the Kalman \ufb01lter and MSSM with \ufb02exible likelihoods\nfrom two sessions with clinical trial participant T10. Left: Box plots of squared error between unit vectors from\ncursor to target and normalized (unit vector) decoder output for each four-minute comparison block in a session.\nMSSM errors are consistently smaller. Right: Two metrics that describe the smoothness of cursor trajectories,\nintroduced by MacKenzie et al. [2001] and commonly used to quantify iBCI performance [Kim et al., 2008,\nSimeral et al., 2011]. The task axis for a trajectory is the straight line from the cursor\u2019s position at the start of a\ntrajectory to a goal. Orthogonal directional changes measure the number of direction changes towards or away\nfrom the goal, and movement direction changes measure the number of direction changes towards or away from\nthe task axis. The MSSM decoder shows signi\ufb01cantly fewer direction changes according to both metrics.\n\n5.2 Realtime evaluation\n\nNext, we examined whether the MSSM method was effective for realtime iBCI control by a clinical\ntrial participant. On two different days, a clinical trial participant (T10) completed six four-minute\ncomparison blocks. In these blocks, we alternated using an MSSM decoder with \ufb02exible likelihoods\nand novel preprocessing, or a standard Kalman decoder. As with the Kalman decoding described in\nJarosiewicz et al. [2015], we used the Kalman \ufb01lter in conjunction with a bias correcting postprocess-\ning heuristic. We used the feature selection method proposed by Malik et al. [2015] to select D = 60\nchannels of neural data, and used these same 60 channels for both decoders.\nJarosiewicz et al. [2015] selected the timesteps of data to use for parameter learning by taking the\n\ufb01rst two seconds of each trajectory after a 0.3s reaction time. For both decoders, we instead selected\nall timesteps in which the cursor was a \ufb01xed distance from the cued goal because we found this\nalternative method lead to improvements in of\ufb02ine decoding. Both methods for selecting subsets of\nthe calibration data are designed to compensate for the fact that vectors from cursor to target are not a\nreliable estimator for participants\u2019 intended aim when the cursor is near the target.\nDecoding accuracy. Figure 6 shows that our MSSM decoder had less directional error than the\ncon\ufb01guration of the Kalman \ufb01lter that we compared to. We con\ufb01rmed the statistical signi\ufb01cance\nof this result using a Wilcoxon rank sum test. To accommodate the Wilcoxon rank sum test\u2019s\nindependence assumption, we divided the data into individual trajectories from a starting point\ntowards a goal, that ended either when the cursor reached the goal or at a timeout (10 seconds). We\nthen computed the mean squared error of each trajectory, where the squared error is the squared\nEuclidean distance between the normalized (unit vector) decoded vectors and the unit vectors from\ncursor to target. Within each session, we compared the distributions of these mean squared errors for\ntrajectories between decoders (p < 10\u22126 for each session). MSSM also performed better than the\nKalman on metrics from MacKenzie et al. [2001] that measure the smoothness of cursor trajectories\n(see Fig. 6).\nFigure 7 shows example trajectories as the cursor moves toward its target via the MSSM decoder or\nthe (bias-corrected) Kalman decoder. Consistent with the quantitative error metrics, the trajectories\nproduced by the MSSM model were smoother and more direct than those of the Kalman \ufb01lter,\nespecially as the cursor approached the goal. The distance ratio (the ratio of the length of the\ntrajectory to the line from the starting position to the goal) averaged 1.17 for the MSSM decoder and\n1.28 for the Kalman decoder, a signi\ufb01cant difference (Wilcoxon rank sum test, p < 10\u22126). Some\ntrajectories for both decoders are shown in Figure 7. Videos of cursor movement under both decoding\nalgorithms, and additional experimental details, are included in the supplemental material.\nDecoding speed. We controlled for speed by con\ufb01guring both decoders to average the same fast\nspeed determined in collaboration with clinical research engineers familiar with the participant\u2019s\npreferred cursor speed. For each decoder, we collected a block of data in which the participant used\n\n8\n\nMSSMKalmanMSSMKalmanMSSMKalman00.511.522.53Squared errorSession 1TimeMSSMKalmanMSSMKalmanMSSMKalman00.511.522.53Squared errorSession 2Time\fFigure 7: Examples of realtime decoding trajectories. Left: 20 randomly selected trajectories for the Kalman\ndecoder, and 20 trajectories for the MSSM decoder. The trajectories are aligned so that the starting position is at\nthe origin and rotated so the goal position is on the positive, horizontal axis. The MSSM decoder exhibits fewer\nabrupt direction changes. Right: The empirical probability of instantaneous angle of movement, after rotating all\ntrajectories from the realtime data (24 minutes of iBCI use with each decoder). The MSSM distribution (shown\nas translucent cyan) is more peaked at zero degrees, corresponding to direct motion towards the goal.\n\nthat decoder to control the cursor. For each of these blocks, we computed the trimmed mean of\nthe speed, and then linearly extrapolated the speed gain needed for the desired speed. Although\nsuch an extrapolation is approximate, the average times to acquire a target with each decoder at the\nextrapolated speed gains were within 6% of each other: 2.6s for the Kalman decoder versus 2.7s for\nthe MSSM decoder. This speed discrepancy is dominated by the relative performance improvement\nof MSSM over Kalman: the Kalman had a 30.7% greater trajectory mean squared error, 249% more\northogonal direction changes, and 224% more movement direction changes.\nThis approach to evaluating decoder performance differs from that suggested by Willett et al. [2016],\nwhich discusses the possibility of optimizing the speed gain and other decoder parameters to minimize\ntarget acquisition time. In contrast, we matched the speed of both decoders and evaluated decoding\nerror and smoothness. We did not extensively tune the dynamics parameters for either decoder,\ninstead relying on the Kalman parameters in everyday use by T10. For MSSM we tried two values of\n\u03b7, which controls the sampling of goal states (6), and chose the remaining parameters of\ufb02ine.\n\n6 Conclusion\n\nWe introduce a \ufb02exible likelihood model and multiscale semi-Markov (MSSM) dynamics for cursor\ncontrol in intracortical brain-computer interfaces. The \ufb02exible tuning likelihood model extends the\ncosine tuning model to allow for multimodal tuning curves and narrower peaks. The MSSM dynamic\nBayesian network explicitly models the relationship between the goal position, the cursor position,\nand the angle of intended movement. Because the goal position changes much less frequently than\nthe angle of intended movement, a decoder\u2019s past knowledge of the goal position stays relevant for\nlonger, and the MSSM model can use longer histories of neural activity to infer the direction of\ndesired movement.\nTo create a realtime decoder, we derive an online variant of the junction tree algorithm with provable\naccuracy guarantees. We demonstrate a signi\ufb01cant improvement over the Kalman \ufb01lter in of\ufb02ine\nexperiments with neural recordings, and demonstrate promising preliminary results in clinical trial\ntests. As seen in the videos, the MSSM decoder yields an appreciably straighter and smoother\ntrajectory than the Kalman decoder. Future work will further evaluate the suitability of this method\nfor clinical use. We hope that the MSSM graphical model will also enable further advances in iBCI\ndecoding, for example by encoding the structure of a known user interface in the set of latent goals.\n\nAuthor contributions DJM, JLP, and EBS created the \ufb02exible tuning likelihood and the multiscale\nsemi-Markov dynamics. DJM derived the inference (decoder), wrote software implementations of\nthese methods, and performed data analyses. DJM, JLP, and EBS designed of\ufb02ine experiments. DJM,\nBJ, and JDS designed clinical research sessions. LRH is the sponsor-investigator of the BrainGate2\npilot clinical trial. DJM, JLP, and EBS wrote the manuscript with input from all authors.\n\n9\n\nGoalNear Goal\fAcknowledgments\n\nThe authors thank Participants T9 and T10 and their families, Brian Franco, Tommy Hosman, Jessica\nKelemen, Dave Rosler, Jad Saab, and Beth Travers for their contributions to this research. Support\nfor this study was provided by the Of\ufb01ce of Research and Development, Rehabilitation R&D Service,\nDepartment of Veterans Affairs (B4853C, B6453R, and N9228C), the National Institute on Deafness\nand Other Communication Disorders of National Institutes of Health (NIDCD-NIH: R01DC009899),\nMGH-Deane Institute, and The Executive Committee on Research (ECOR) of Massachusetts General\nHospital. The content is solely the responsibility of the authors and does not necessarily represent the\nof\ufb01cial views of the National Institutes of Health, or the Department of Veterans Affairs or the United\nStates Government. CAUTION: Investigational Device. Limited by Federal Law to Investigational\nUse.\nDisclosure: Dr. Hochberg has a \ufb01nancial interest in Synchron Med, Inc., a company developing\na minimally invasive implantable brain device that could help paralyzed patients achieve direct\nbrain control of assistive technologies. Dr. Hochberg\u2019s interests were reviewed and are managed by\nMassachusetts General Hospital, Partners HealthCare, and Brown University in accordance with their\ncon\ufb02ict of interest policies.\n\nReferences\nA. B. Ajiboye, F. R. Willett, D. R. Young, W. D. Memberg, B. A. Murphy, J. P. Miller, B. L. Walter,\nJ. A. Sweet, H. A. Hoyen, M. W. Keith, et al. Restoration of reaching and grasping movements\nthrough brain-controlled muscle stimulation in a person with tetraplegia. The Lancet, 389(10081):\n1821\u20131830, 2017.\n\nB. Amirikian and A. P. Georgopulos. Directional tuning pro\ufb01les of motor cortical cells. J. Neurosci.\n\nRes., 36(1):73\u201379, 2000.\n\nD. Bacher, B. Jarosiewicz, N. Y. Masse, S. D. Stavisky, J. D. Simeral, K. Newell, E. M. Oakley, S. S.\nCash, G. Friehs, and L. R. Hochberg. Neural point-and-click communication by a person with\nincomplete locked-in syndrome. Neurorehabil Neural Repair, 29(5):462\u2013471, 2015.\n\nX. Boyen and D. Koller. Tractable inference for complex stochastic processes. In Proceedings of\n\nUAI, pages 33\u201342. Morgan Kaufmann Publishers Inc., 1998.\n\nJ. L. Collinger, B. Wodlinger, J. E. Downey, W. Wang, E. C. Tyler-Kabara, D. J. Weber, A. J.\nMcMorland, M. Velliste, M. L. Boninger, and A. B. Schwartz. High-performance neuroprosthetic\ncontrol by an individual with tetraplegia. The Lancet, 381(9866):557\u2013564, 2013.\n\nJ. A. Donoghue, J. H. Bagley, V. Zerris, and G. M. Friehs. Youman\u2019s neurological surgery: Chapter\n\n94. Elsevier/Saunders, 2011.\n\nG. W. Fraser, S. M. Chase, A. Whitford, and A. B. Schwartz. Control of a brain\u2013computer interface\n\nwithout spike sorting. J. Neuroeng, 6(5):055004, 2009.\n\nA. P. Georgopoulos, J. F. Kalaska, R. Caminiti, and J. T. Massey. On the relations between the\ndirection of two-dimensional arm movements and cell discharge in primate motor cortex. J.\nNeurosci., 2(11):1527\u20131537, 1982.\n\nV. Gilja, C. Pandarinath, C. H. Blabe, P. Nuyujukian, J. D. Simeral, A. A. Sarma, B. L. Sorice, J. A.\nPerge, B. Jarosiewicz, L. R. Hochberg, et al. Clinical translation of a high performance neural\nprosthesis. Nature Medicine, 21(10):1142, 2015.\n\nL. R. Hochberg, M. D. Serruya, G. M. Friehs, J. A. Mukand, M. Saleh, A. H. Caplan, A. Branner,\nD. Chen, R. D. Penn, and J. P. Donoghue. Neuronal ensemble control of prosthetic devices by a\nhuman with tetraplegia. Nature, 442(7099):164\u2013171, 2006.\n\nL. R. Hochberg, D. Bacher, B. Jarosiewicz, N. Y. Masse, J. D. Simeral, J. Vogel, S. Haddadin, J. Liu,\nS. S. Cash, P. van der Smagt, et al. Reach and grasp by people with tetraplegia using a neurally\ncontrolled robotic arm. Nature, 485(7398):372, 2012.\n\n10\n\n\fB. Jarosiewicz, A. A. Sarma, D. Bacher, N. Y. Masse, J. D. Simeral, B. Sorice, E. M. Oakley, C. Blabe,\nC. Pandarinath, V. Gilja, et al. Virtual typing by people with tetraplegia using a self-calibrating\nintracortical brain-computer interface. Sci. Transl. Med., 7(313):313ra179\u2013313ra179, 2015.\n\nS. P. Kim, J. D. Simeral, L. R. Hochberg, J. P. Donoghue, and M. J. Black. Neural control of computer\ncursor velocity by decoding motor cortical spiking activity in humans with tetraplegia. J. Neuroeng,\n5(4):455, 2008.\n\nS. Koyama, S. M. Chase, A. S. Whitford, M. Velliste, A. B. Schwartz, and R. E. Kass. Comparison\nof brain\u2013computer interface decoding algorithms in open-loop and closed-loop control. J Comput\nNeurosci, 29(1-2):73\u201387, 2010.\n\nP. Levy. Semi-Markovian processes. Proc: III ICM (Amsterdam), pages 416\u2013426, 1954.\n\nI. S. MacKenzie, T. Kauppinen, and M. Silfverberg. Accuracy measures for evaluating computer\n\npointing devices. In Proceedings of CHI, pages 9\u201316. ACM, 2001.\n\nW. Q. Malik, L. R. Hochberg, J. P. Donoghue, and E. N. Brown. Modulation depth estimation and\nvariable selection in state-space models for neural interfaces. IEEE Trans Biomed Eng, 62(2):\n570\u2013581, 2015.\n\nK. Murphy and Y. Weiss. The factored frontier algorithm for approximate inference in DBNs. In\n\nProceedings of UAI, pages 378\u2013385. Morgan Kaufmann Publishers Inc., 2001.\n\nK. P. Murphy. Dynamic Bayesian networks: Representation, inference and learning. PhD thesis,\n\nUniversity of California, Berkeley, 2002.\n\nC. Pandarinath, P. Nuyujukian, C. H. Blabe, B. L. Sorice, J. Saab, F. R. Willett, L. R. Hochberg, K. V.\nShenoy, and J. M. Henderson. High performance communication by people with paralysis using\nan intracortical brain-computer interface. eLife, 6:e18554, 2017.\n\nJ. D. Simeral, S. P. Kim, M. Black, J. Donoghue, and L. Hochberg. Neural control of cursor trajectory\nand click by a human with tetraplegia 1000 days after implant of an intracortical microelectrode\narray. J. Neuroeng, 8(2):025027, 2011.\n\nW. L. Smith. Regenerative stochastic processes. Proceedings of the Royal Society of London A:\n\nMathematical, Physical and Engineering Sciences, 232(1188):6\u201331, 1955.\n\nF. R. Willett, C. Pandarinath, B. Jarosiewicz, B. A. Murphy, W. D. Memberg, C. H. Blabe, J. Saab,\nB. L. Walter, J. A. Sweet, J. P. Miller, et al. Feedback control policies employed by people using\nintracortical brain\u2013computer interfaces. J. Neuroeng, 14(1):016001, 2016.\n\nF. R. Willett, B. A. Murphy, W. D. Memberg, C. H. Blabe, C. Pandarinath, B. L. Walter, J. A. Sweet,\nJ. P. Miller, J. M. Henderson, K. V. Shenoy, et al. Signal-independent noise in intracortical brain\u2013\ncomputer interfaces causes movement time properties inconsistent with Fitts\u2019 law. J. Neuroeng, 14\n(2):026010, 2017.\n\nS. Z. Yu. Hidden semi-Markov models. Arti\ufb01cial intelligence, 174(2):215\u2013243, 2010.\n\n11\n\n\f", "award": [], "sourceid": 567, "authors": [{"given_name": "Daniel", "family_name": "Milstein", "institution": "Brown University"}, {"given_name": "Jason", "family_name": "Pacheco", "institution": "Brown University"}, {"given_name": "Leigh", "family_name": "Hochberg", "institution": "Brown, MGH, VA, Harvard"}, {"given_name": "John", "family_name": "Simeral", "institution": "Brown University"}, {"given_name": "Beata", "family_name": "Jarosiewicz", "institution": "Stanford University"}, {"given_name": "Erik", "family_name": "Sudderth", "institution": "University of California, Irvine"}]}