{"title": "Kernel-ARMA for Hand Tracking and Brain-Machine interfacing During 3D Motor Control", "book": "Advances in Neural Information Processing Systems", "page_first": 1489, "page_last": 1496, "abstract": "Using machine learning algorithms to decode intended behavior from neural activity serves a dual purpose. First, these tools can be used to allow patients to interact with their environment through a Brain-Machine Interface (BMI). Second, analysis of the characteristics of such methods can reveal the significance of various features of neural activity, stimuli and responses to the encoding-decoding task. In this study we adapted, implemented and tested a machine learning method, called Kernel Auto-Regressive Moving Average (KARMA), for the task of inferring movements from neural activity in primary motor cortex. Our version of this algorithm is used in an on-line learning setting and is updated when feedback from the last inferred sequence become available. We first used it to track real hand movements executed by a monkey in a standard 3D motor control task. We then applied it in a closed-loop BMI setting to infer intended movement, while arms were restrained, allowing a monkey to perform the task using the BMI alone. KARMA is a recurrent method that learns a nonlinear model of output dynamics. It uses similarity functions (termed kernels) to compare between inputs. These kernels can be structured to incorporate domain knowledge into the method. We compare KARMA to various state-of-the-art methods by evaluating tracking performance and present results from the KARMA based BMI experiments.", "full_text": "Kernel-ARMA for Hand Tracking and\n\nBrain-Machine Interfacing During 3D Motor Control\n\nLavi Shpigelman1 , Hagai Lalazar 2 and Eilon Vaadia 3\n\nInterdisciplinary Center for Neural Computation\n\nThe Hebrew University of Jerusalem, Israel\n\n1shpigi@gmail.com, 2hagai@alice.nc.huji.ac.il,\n\n3eilonv@ekmd.huji.ac.il\n\nAbstract\n\nUsing machine learning algorithms to decode intended behavior from neural ac-\ntivity serves a dual purpose. First, these tools allow patients to interact with\ntheir environment through a Brain-Machine Interface (BMI). Second, analyzing\nthe characteristics of such methods can reveal the relative signi\ufb01cance of various\nfeatures of neural activity, task stimuli, and behavior. In this study we adapted, im-\nplemented and tested a machine learning method called Kernel Auto-Regressive\nMoving Average (KARMA), for the task of inferring movements from neural ac-\ntivity in primary motor cortex. Our version of this algorithm is used in an online\nlearning setting and is updated after a sequence of inferred movements is com-\npleted. We \ufb01rst used it to track real hand movements executed by a monkey in\na standard 3D reaching task. We then applied it in a closed-loop BMI setting to\ninfer intended movement, while the monkey\u2019s arms were comfortably restrained,\nthus performing the task using the BMI alone. KARMA is a recurrent method\nthat learns a nonlinear model of output dynamics.\nIt uses similarity functions\n(termed kernels) to compare between inputs. These kernels can be structured to\nincorporate domain knowledge into the method. We compare KARMA to various\nstate-of-the-art methods by evaluating tracking performance and present results\nfrom the KARMA based BMI experiments.\n\n1 Introduction\n\nPerforming a behavioral action such as picking up a sandwich and bringing it to one\u2019s mouth is a\nmotor control task achieved easily every day by millions of people. This simple action, however,\nis impossible for many patients with motor de\ufb01cits. In the future, patients with enough cortical\nactivity remaining may bene\ufb01t from Brain Machine Interfaces that will restore motor control with\nagility, precision, and the degrees of freedom comparable to natural movements. Such high quality\nBMI\u2019s are not yet available. The BMI framework involves recording neural activity, typically using\nchronically implanted electrodes, which is fed in real-time to a decoding algorithm. Such algorithms\nattempt to infer the subject\u2019s intended behavior. The algorithm\u2019s predictions can be used to arti\ufb01-\ncially control an end-effector: a cursor on a screen, a prosthetic arm, a wheelchair, or the subject\u2019s\nown limbs by stimulation of their muscles. This study focuses on the algorithmic component.\nMotor control is a dynamic process involving many feedback loops, relevant time frames, and con-\nstraints of the body and neural processing. Neural activity in primary motor cortex (MI) is part of\nthis process. An early approach at decoding movement from MI activity for BMI (see [1]) was rather\nsimplistic. Instantaneous velocity of the cursor, across a set of movements, was linearly regressed\nagainst neuronal spike rates. This algorithm (known as the Population Vector Algorithm) is equiva-\nlent to modelling each neuron as a consine function of movement velocity. This method is still used\ntoday for BMI\u2019s [2], and has become the standard model in many studies of encoding and learning in\nMI. Our understanding of motor cortex has progressed, and many other factors have been shown to\n\n1\n\n\fcorrelate with neuronal activity, but are typically overlooked in modeling. For example, MI activity\nhas been shown to encode arm posture [3], the dynamic aspects of the movement (such as current\nacceleration, or interaction forces) and the interactions between neurons and their dynamics [4].\nState-of-the-art movement decoding methods typically involve improved modeling of behavior, neu-\nral activity, and the relations between them. For example, Kalman \ufb01ltering (see [5]) has been used to\nmodel the system state as being comprised of current hand position, velocity and acceleration. Thus,\nthe hand movement is assumed to have roughly constant acceleration (with added Gaussian noise\nand, consequently, minimal jerk) and the neural activity is assumed to be a linear function of the\nhand state (with added Gaussian noise). Particle \ufb01ltering, which relaxes some of the linearity and\nGaussian assumptions, has also been applied in an of\ufb02ine setting (see [6]). Support Vector Regres-\nsion (SVR) from neural activity to current hand velocity (see [7]) has the advantage of allowing for\nextraction of nonlinear information from neuronal interactions, but is missing a movement model.\nOne of our previous studies ([8]) combines a linear movement model (as in Kalman \ufb01ltering) with\nSVR-based nonlinear regression from neural activity.\nKARMA (see [9] for one of its \ufb01rst appearances, or [10] for a more recent one) is a kernelized\nversion of the ARMA method [11]. It performs ARMA in a kernel-induced feature space (for a\ncomprehensive explanation of this kernel-trick, see [12]). It estimates the next system state as a\nfunction of both the time window of previous state estimates (the Auto-Regressive part) and the\ntime window of previous observations (the Moving-Average part). In our application, we extend\nits formulation to the Multi-Input Multi-Output (MIMO) case, allowing for better modeling of the\nsystem state. We apply it in an online learning paradigm, and by limiting the number of support\nvectors turn it into an adaptive method. This allows for real-time inference, as is necessary for BMI.\nIn section 2 we explain the lab setup, the problem setting, and introduce our notation. In section 3\nwe describe KARMA, and our online and adaptive version of it, in detail. We explain the available\nmodeling options and how they can be used to either improve performance or to test ideas regarding\nmotor control and neural encoding. Section 4 describes KARMA\u2019s performance in tracking hand\nmovements and compares it with other state-of-the-art methods. Section 5 presents results from our\nBMI experiment using KARMA and, \ufb01nally, we summarize in section 6\n\n2 Lab setup and problem setting\n\nIn our experiments, a monkey performed a visuomotor control task that involved moving a cursor\non a screen from target to target in 3D virtual space. Neuronal activity from a population of single\nand multi-units was recorded with a chronically implanted array of 96 electrodes (Cyberkinetics,\nInc.) in MI, and used to calculate spike rates (spike counts in 50ms time bins smoothed by a causal\n\ufb01lter). In hand-control (open-loop) mode, the monkey used its hand (continuously tracked by an\noptical motion capture system; Phoenix Tech., Inc.) to move the cursor. Data collected from these\nsessions is used here to assess algorithm performance, by using the real arm movements as the\ntarget trajectories. In hand-control, behavior is segmented into sequences of continuous recordings,\nseparated by time periods during which the monkey\u2019s hand is not in view of the tracking device (e.g.\nwhen the monkey stops to scratch itself). Each sequence is made up of target-to-target reaching trials\n(some successful and some not). The targets appeared randomly in the 27 corners, centers of faces,\nand middle of a virtual cube whose side was 6cm. The target radii were 2.4cm. A successful trial\nconsisted of one reaching movement that started at rest in one target and ended at rest in the next\ntarget (with required target hold periods during which the cursor must not leave the target). The next\ntarget appears at some point during the hold period of the previous target. Food reward is provided\nthrough a tube after each success. In case of failure, the cursor disappears for 1-2 seconds (failure\nperiod). During this time the monkey\u2019s hand is still tracked. In the BMI (closed-loop) setting, the\nmonkey\u2019s hands were comfortably restrained and the KARMA algorithm\u2019s inference was used to\nmove the cursor. Trial failures occured if the reaching movement took longer than 6 seconds or if\nthe cursor was not kept inside the target during a 0.25s hold period. During trial-failure periods the\ninference was stopped and at the next trial the cursor reappeared where it left off. The trial-failure\nperiod was also used to pass the latest recorded trial sequence to the model-learning process, and to\nreplace the working model with an updated one, if available.\nIn this text, X (Capital, bold) is a matrix, x (bold) is a vector ( so is xi, xt or xi\nt) and x is a scalar.\nt \u2208 IRq to designate the neural activity (of q cortical\n(x)T signi\ufb01es transposition. We will use xi\nunits) at time bin t in behavior sequence i, which we refer to as observations. Given a window size,\n\n2\n\n\f(cid:1)T\n\n(cid:16)(cid:0)xi\n\n, . . . ,(cid:0)xi\n\n(cid:1)T(cid:17)T \u2208 IRsq is an sq-long vector comprising a concatenated\n\nt\n\nt =\n\nt\u2212s+1\n\nt\u2212r:t\u22121\n\nt\u2212s+1:t =\n\n(cid:1)T(cid:0)xi\n\nwhere tfi is the number of steps in the whole ith trajectory. Similarly, yi\n\nt\u2212s+1:t\nt to concatenate true (rather than estimated) state values.\n\n(cid:16)(cid:0)\u02c6yi\ntrajectories: (cid:8)xi, yi(cid:9)n\n\n(cid:1)T(cid:17)T \u2208 IRrd+sq to concatenate windows of estimated\n\ns, xi\nwindow of observations ending at time t, of trajectory i. xi will be short-hand notation meaning\nt \u2208 IRd are used\nxi\n1:tfi\nto designate cursor position (d = 3). We refer to yi\nt as the state trajectory. Given a window size,\nt\u2212r:t\u22121 \u2208 IRrd is a concatenated vector of states. Estimated states are differentiated from true\nr ,yi\nstates (or desired states, as will be explained later) by addition of a hat: \u02c6yi\nt. Furthermore (given s and\nr) we will use \u02c6vi\nstates and of neural observations and vi\nIn the hand-control setting, we are given a (fully observed) data-set of neural activities and state\ni=1. Our goal is to learn to reconstruct the state trajectories from the neural\nactivities. We adhere to the online learning paradigm in which at each step, i, of the process we are\ngiven one observation sequence, xi, predict \u02c6yi, then receive the true yi and update the model. This\nallows the model to adapt to changes in the input-output relation that occurs over time.\nIn BMI mode, since hand movements were not per-\nformed, we do not know the correct cursor movement.\nInstead, during learning we use the cursor movement\ngenerated in the BMI and the positions of the targets\nthat the monkey was instructed to reach to guess a\ndesired cursor trajectory which is used to replace the\nmissing true trajectory as feedback. The illustration\non the right shows the BMI setup from an algorithmic\npoint of view.\n\n3 KARMA, modeling options, and online learning\n\n(cid:80)s\n\nt = (cid:80)r\n\nt\n\nk=1 Akyi\n\nt = W\u02c6vi\n\nl=1 Bl xi\n\nt\u2212l+1 + ei\n\nk=1 and {Bl}s\n\nt, where {Ak}r\n\nt (see notation section).\n\nAs stated earlier, KARMA is a kernelized ARMA. In ARMA: yi\n\nt\u2212k +\nl=1 are the respective Auto-Regressive (AR) and\nMoving Average (MA) parameters and ei\nt are residual error terms. Given these model parameters\nand initial state values, the rest of the state trajectory can be estimated from the observations by\nrecursive application, replacing true state values with the estimated ones. Thus, ARMA inference is\nessentially application of a linear (MIMO) IIR \ufb01lter. De\ufb01ning W = [Ar, . . . , A1, Bs, . . . , B1], the\nnext state estimate is simply \u02c6yi\nKernelizing ARMA involves application of the kernel trick. A kernel function k (v1, v2) :\nIRrd+sq \u00d7 IRrd+sq \u2192 IR is introduced, which, conceptually, can be viewed as a dot product of\nfeature vectors: k (v1, v2) = \u03c6T (v1) \u03c6 (v2) where the features are possibly complicated functions\nof both states and (neural) observations. Inference takes the form \u02c6yi\n\u03c4 \u2208 IRd are learned weight vectors and vj\n\u03b1j\nset. Conceptually, KARMA inference can be viewed as \u02c6yi\nwith ARMA, \u02c6vi\n\u03c4 ) and\neach recursive step of KARMA is linear regression in the feature space of observations + states. The\n\n(cid:1) where\n\u03c4 k(cid:0)\u02c6vi\nt =(cid:80)\nt = W\u03c6\u03c6(cid:0)\u02c6vi\n(cid:1) where, as compared\nt is replaced by its feature vector, W is replaced by W\u03c6 = (cid:80)\n(cid:9) are learned so as to solve the following optimization problem (presented in its primal\n(cid:12)(cid:12)\u0001\n(cid:1)(cid:1)\n\nweights,(cid:8)\u03b1j\nform): arg minW\u03c6 (cid:107)W\u03c6(cid:107)2 + c(cid:80)\n\nab is\nthe Frobenius matrix norm, the sum in the second term is over all trials, times and state dimensions\nof the examples in the training set, |v|\u0001 = max{0,|v|\u2212\u0001} is the \u0001-insensitive absolute error and c is a\nconstant that determines the relative trade-off between the \ufb01rst (regularization) term and the second\n(error) term. Note that during learning, the states are estimated using the true / desired previous state\nvalues as input instead of the estimated ones (contrary to what is done during inference). \u201cLuckily\u201d,\nthis optimization problem reduces to standard SVR where xi\nt. This replacement\ncan be done as a preprocessing step in learning and a standard SVR solver can then be used to \ufb01nd\nthe weights. Inference would require plugging in the previously estimated state values as part of the\ninputs between iterative calls to SVR inference.\nApplication of KARMA to a speci\ufb01c domain entails setting of some key hyper-parameters that\nmay have drastic effects on performance. The relatively simple ones are the window sizes (r and\n\n, where (cid:107)w(cid:107)2 = (cid:80)\n\n\u03c4 are examples from a training set, known as the support\n\nk \u2212(cid:0)W\u03c6\u03c6(cid:0)vi\n\nt is replaced with vi\n\n(cid:12)(cid:12)(cid:0)yi\n\nt\n\nj,\u03c4 \u03b1j\n\n\u03c4 \u03c6T (vj\n\na,b (W)2\n\nj,\u03c4 \u03b1j\n\nt, vj\n\u03c4\n\n(cid:1)\n\n\u03c4\n\ni,t,k\n\nt\n\nk\n\n3\n\n\ftime series. At learning step k, all of(cid:8)xi, yi(cid:9)k\n\ns) and trade-off parameter c. Beyond the necessary selection of the cursor trajectories as the states,\naugmenting state dimensions (whose values are known at training and inferred during model testing)\ncan be added in order to make the model use them as explicit features. This idea was tried in our\nhand tracking experiments using features such as absolute velocity and current trial state (reaching\ntarget, holding at target and trial-failure time). But since results did not improve signi\ufb01cantly, we\ndiscontinued this line of research. The kernel function and its parameters must also be chosen.\nNote that the kernel in this algorithm is over structured data, which opens the door to a plethora\nof choices. Depending on one\u2019s view this can be seen as an optimization curse or as a modeling\nblessing. It obviously complicates the search for effective solutions but it allows to introduce domain\nknowledge (or assumptions) into the problem. It can also be used as a heuristic for testing the relative\ncontribution of the assumptions behind the modeling choices. For example, by choosing r = 0 the\nalgorithm reduces to SVR and the state model (and its dynamics) are ignored. By selecting a kernel\nwhich is a linear sum of two kernels, one for states and one for observations, the user assumes that\nstates and observations have no \u201csynergy\u201d (i.e. each series can be read without taking the other into\naccount). This is because summing of kernels is equivalent to calculating the features on their inputs\nseparately and then concatenating the feature vectors. Selecting linear kernels reduces KARMA to\nARMA (using its regularized loss function).\nIn online learning, one may change the learned model between consecutive inferences of (whole)\ni=1 are available for training. A naive solution would\nbe to re-learn the model at every step, however, this would not be the best solution if one believes\nthat the source of the input-output relation is changing (for example, in our BMI, cortical units may\nchange their response properties, or units may appear or disappear during a recording session). Also,\nit may not be feasible to store all the sequences, or learning may take too long (opening up a delay\nbetween data acquisition until a new model is ready). If the resulting model has too many support\nvectors, too much time is required for each inference step (which is less than 50ms in our BMI\nsetup). We deal with all the issues above by limiting the number of examples (vi\nt) that are kept in\nmemory (to 5000 in hand-control tracking and 3000 for real-time use in the BMI). At each online\nlearning iteration, the latest data is added to the pool of examples one example at a time, and if\nthe limit has been reached another example is selected at random (uniformly over the data-set) and\nthrown out. This scheme gives more mass to recent observations while allowing for a long tail of\nolder observations. For a 3000 sized database and examples coming in at the rate of one per 50ms,\nthe cache is \ufb01lled after the \ufb01rst 150 seconds. Afterwards, the half life (the time required for an\nexample to have a 50% chance of being thrown out) of an example is approximately 104 seconds,\nor conversely, at each point, approx. 63% of the examples in the database are from the last 150\nseconds and the rest are earlier ones. This scheme keeps the inference time to a constant and seems\nreasonable in terms of rate of adaptation. We chose 5000 for the tracking (hand-control) experiments\nsince in those experiments there is no real-time inference constraint and the performance improves\na bit (suggesting that the 3000 size is not optimal in terms of inference quality). The similarity\nbetween consecutive examples is rather high as they share large parts of their time windows (when\nARMA parameters r or s are large). Throwing away examples at random has a desired effect of\nlessening the dependency between remaining examples.\n\n4 Open-loop hand tracking testing\n\nTo test various parametrization of KARMA and to compare its performance to other methods we\nused data from 11 hand-control recording days. These sessions vary in length from between 80\nto 120 minutes of relatively continuous performance on the part of the monkey. Success rates in\nthis task were at the 65-85% range. Cortical units were sorted in an automated manner every day\nwith additional experimenter tuning. The number of very well-isolated single units ranged between\n21-41. The remaining units consisted of 140-150 medium quality and multi-units, which the sorting\nsoftware often split into more than one channel.\nMost of the different algorithms that are compared here have free hyper-parameters that need to be\nset (such as a Gaussian kernel width for spike rates, the maximal informative time window of neural\nactivities, s and the c trade-off parameter). We had a rough estimate for some of these from previous\nexperiments using similar data (see [8]). To \ufb01ne-tune these parameters, a brute-force grid search was\nperformed on data from one of the 11 sessions in a (batch) 5-fold cross validation scheme. Those\nparameters were then kept \ufb01xed.\n\n4\n\n\ft = W\u03c8\u03c8(\u02c6yi\n\nt\u2212r:t\u22121) + W\u03c6\u03c6(xi\n\nEarlier experiments showed the Gaussian kernel to be a good candidate for comparing neural spike\nrate vectors. It can also be calculated quickly, which is important for the BMI real-time constraint.\nWe tested several variations of structured kernels on neuro-movement inputs. These variations con-\nsisted of all combinations of summing or multiplying Gaussian or linear kernels for the spike rates\nand movement states. Taking a sum of Gaussians or their product produced the best results (with\nno signi\ufb01cant difference between these two options). We chose the sum (having the conceptual in-\nference form: \u02c6yi\nt\u2212s:t) where \u03c8, \u03c6 are the in\ufb01nite feature vectors\nof the Gaussian kernel). The next best result was achieved by summing a Gaussian spike rate ker-\nnel and a linear movement kernel (which we will call lin-y-KARMA). The sum of linear kernels\nproduces ARMA (which we also tested). The test results that are presented in this study are only\nfor the remaining 10 recording sessions. The main performance measure that we use here is the\n(Pearson) correlation coef\ufb01cient (CC) between true and estimated values of positions (in each of the\n3 movement dimensions). To gauge changes in prediction quality over time we use CC in a running\nwindow of sequences (window size is chosen so as to decrease the noise in the CC estimate). In\nother cases, the CC for a whole data-set is computed.\nTo illustrate KARMA\u2019s performance we provide a movie (see video 1 in supplementary material)\nshowing the true hand position (in black) and KARMA\u2019s tracking estimate (in blue) during a con-\ntinuous 150 second sequence of target-to target reach movements. This clip is of a model that was\nlearned in an online manner using the previous (180) sequences, using a support vector cache of\n3000 (as described in section 3). The initial position of the cursor is not given to the algorithm.\nInstead the average start positions in previous sequences is given as the starting point. The CC in\nthe time window of the previous 40 sequences (0.9, 0.92 and 0.96 for the 3 dimensions) is given to\nprovide a feeling of what such CC\u2019s look like. Similarly, Figure 1.B shows tracking and true position\nvalues for an 80 second segment towards the end of a different session.\nKARMA achieves good performance and it does so with relatively small amounts of data. Figure\n1.A shows tracking quality in terms of a running window of CC\u2019s over time. CC\u2019s for the \ufb01rst\nsequences are calculated on predictions made up to those times. While these CC\u2019s are more noisy it\nis clear that a useful model is reached within the \ufb01rst 3 minutes (CC\u2019s all higher than 0.6) and close\nto peak performance is already available within the \ufb01rst 10 minutes.\n\nFigure 1: KARMA performance: (A) Correlation coef\ufb01cients in a running window of 20 sequences (or less\nat the session start) for a whole 95 minute session (mean CC window size is 9.7 minutes). (B) True (gray)\ntracked positions in an 80 second segment at minute 90 of the session. (C) Effect of loosing recording\nvs.\nelectrodes: tracking was performed over a full recording session using randomly chosen subsets of electrodes.\nFor each selected number of electrodes (from the full 92 available down to 5 electrodes) 50 repetitions were\nrun. CC\u2019s were calculated per run over the last two thirds of the session (to avoid effects of initial performance\ntransients) and averaged over the 3 movement dimensions. Their distributions (over repetitions) are shown\nin terms of the median CC (red center line), quartile values (skirt containing 50% of the CC\u2019s) and extremal\nvalues (whiskers) for each number of electrodes. (D) Effect of delay time between training data and test data:\nFor the session shown in (A), marked \u2019day 1\u2019 and for another session (marked \u2019day 2\u2019), hand movement in 20\nminute time windows towards the session ends were reconstructed in an online manner but instead of using\nthe same sequences as training data (with a 1 step delay), earlier sequences were used. Figure (A) shows the\ntime window that corresponded to opening a maximal time difference of 60 minutes between the last inferred\nsequence (at minute 90) and the last learned sequence (at minute 30). CC\u2019s for the test windows (averaged over\nmovement dimensions) are shown as a function of delay time for the two days.\n\n5\n\nCDCDABearliestdelay traindelaystestpositionsABearliestdelay traindelaystestpositions\fFigure 2: Algorithm comparisons: 10 hand-movement sessions were each divided into 3 equally long blocks\nof 25-35 minutes (the last few minutes were discarded since during this time the monkey often stopped paying\nattention to the task) to create 30 data-sets. The following algorithms were run on each data-set in an online\nmanner: KARMA, lin-y-KARMA, ARMA and SVR. All four were implemented as versions of the KARMA\nby varying its parameters. In all cases a support vector cache of 5000 was enforced as described in section 4.\nA Kalman Filter was also implemented so as to allow for a window of observations as input and a window of\nmovement positions as the state (this version was previously shown to outperform the standard Kalman Filter\nwhich has r = s = 1). It was also learned in an online manner, replacing inverses with pseudo-inverses where\nnecessary to avoid non-invertible matrices when data-sets are too small. Results are shown as scatter plots of\nCC values (30 data-sets and 3 movement dimensions produce 90 points per scatter plot). Each point compares\nKARMA to another algorithm in a speci\ufb01c data-set and movement dimension pair. Points above the diagonal\nmean a higher score for KARMA. The Graph on the left shows win-scores for each pair of algorithm. Win-\nscore is de\ufb01ned at the percentage of times one algorithm got a higher CC than another. Edge direction points\nto the loser. The movement reconstruction on the right (horizontal position only) shows KARMA vs. SVR in a\nsample 18 second window.\n\nProbably the highest source of variability in BMI performance across subjects is the location and\nnumber of electrodes in the brain. To test how performance with KARMA would degrade if elec-\ntrodes were lost we simulated an electrode dropping experiment (see \ufb01gure 1.C). Let\u2019s consider a\nCC of 0.7 as a minimal reasonable performance quality. Let\u2019s also assume that with 50 repetitions,\nminimal values roughly represent a worst case scenario in terms of mishaps that do not involve\nmovement of the whole array. Then it seems that we can get by easily with only a third of the\narray (28 electrodes) operational. In terms of medians (average bad luck) we could do with less.\nMaximal values are relevant in case we need to choose the good electrodes. This may be relevant\nin situations involving implanted chips that extract and wirelessly transmit neural activity and may\nhave constraints in terms of energy expenditure or bandwidth.\nMost BMI experiments (e.g. [2, 13] with the exception of [1]) use \ufb01xed models that are learned\nonce at the beginning of a session. Our version of KARMA is adaptive.\nIn order to check the\nimportance of adapting to changes in the recorded neural activity we ran an experiment in which\nvariable delays were opened between the inference times and the latest available data for learning.\n\ni.e. after inference of sequence yi from xi , sequence pair(cid:8)xi\u2212k, yi\u2212k(cid:9) where k > 0 was \ufb01rst\n\nmade available. Figure 1.D shows a degradation in performance during the test periods as the delay\ngrows for two recording sessions. This suggests that adaptability of the algorithm is important for\nkeeping high performance levels. There are two possible reasons for the observed degradation. One\nis changes in the neural activity within the brain. The other is changes in the process that extracts\nthe neural activity in the BMI (e.g. electrode movements). Differentiating between the two options\nis a subject of future work. In the BMI setting, feedback is involved. The subject might be able to\neffortlessly modulate his neural activity and keep it in good \ufb01t with the algorithm. In section 5 we\naddress this issue by running BMI sessions in which the model was frozen.\nComparison of KARMA to other methods is shown in \ufb01gure 2. It is clear that KARMA performs\nmuch better than ARMA and the Kalman Filter, suggesting that a nonlinear interpretation of neural\nactivity is helpful. While KARMA is statistically signi\ufb01cantly better than SVR, the differences in\nCC values are not very big (note the scaling difference of the scatter plots). Looking at the movement\nreconstruction comparison it seems that SVR has a good average estimate of the current position,\n\n6\n\nKARMAARMAKalmanFilterSVR95.6%Above diagonal is better\ft = A\u02c6yi\n\nt\u2212s:t)).\n\nt = W\u03c6\u03c6(xi\n\nhowever, missing a movement model (SVR has the form \u02c6yi\nt\u2212s:t) ) it \ufb02uctuates rapidly\naround the true value. This \ufb02uctuation may not be very apparent in the CC values however it would\nmake a BMI much more dif\ufb01cult to control. Lin-y-KARMA uses a linear movement model (and\nhas the form: \u02c6yi\nIts performance is inferior to full KARMA.\nHaving a nonlinear movement model means that different areas of the state-space get treated in\nlocally relevant fashion. This might explain why full KARMA outperforms. Note that the difference\nbetween lin-y-KARMA and SVR are not very signi\ufb01cant (win-score of only 65.6%). Comparison\nto the Population Vector algorithm was also done however the PVA achieved especially bad results\nfor our long sequences since it accumulates errors without any decay (this is less of a problem in\nBMI since the subject can correct accumulated errors ). We therefore omit showing them here.\n\nt\u2212r:t\u22121 + W\u03c6\u03c6(xi\n\nFigure 3: All graphs show success rates. The light, noisy plots are success rates in consecutive bins of 10 trials\nwhile the smoother plots are the result of running a smoothing \ufb01lter on the noisy plots. Mixing mode was used\non day 1 and part of day 2. Afterwards we switched to full BMI (mixing factor 100%). Hands were allowed\nto move freely until day 4. On day 4 both hands were comfortably restrained for the \ufb01rst time and though\nperformance levels dropped, the monkey did not attempt to move its hands. On day 5 an attempt to freeze the\nmodel was made. When performance dropped and the monkey became agitated we restarted the BMI from\nscratch and performance improved rapidly. Day 6 consists of full BMI but with the targets not as far apart as\nwith hand-control. This makes the task a bit easier and allowed for higher success rates. On days 7 and 8 a\nBMI block was interleaved with hand control blocks. Only the BMI blocks are shown in the top graph. The full\nthree blocks of day 8 are shown in the bottom right graph. Bottom left graph shows a recording session during\nwhich the model was frozen.\n\n5 BMI experiment\n\nThe BMI experiment was conducted after approximately four months of neural recordings during\nwhich the monkey learned and performed the hand-control task. Figure 3 shows a trace of the\n\ufb01rst days of the BMI task. A movie showing footage from these days is in the supplementary\nmaterial (video 2). To make the transition to BMI smooth, the \ufb01rst 1.5 days consisted of work in\na mixed mode, during which control of the cursor was a linear combination of the hand position\nand KARMA\u2019s predictions. We did this in order to see if the monkey would accept a noisier control\nscheme than it was used to. During the next 1.5 days the cursor was fully controlled by KARMA,\nbut the monkey kept moving as if it was doing hand-control. i.e. it made movements and corrections\nwith its hands. On day 4 the monkey\u2019s free hand was comfortably restrained. Despite our concerns\nthat the monkey would stop performing it seemed impervious to this change, except that control over\nthe cursor became more dif\ufb01cult. On days 5 and 6 we made some tweaks to the algorithm (tried to\nfreeze learning and change the way in which correct movement trajectories are generated) and the\n\n7\n\n020406080100Success ratehand movingpure BMI controlmixed modeboth hands restrainedmixingfactor:0\u2212>50%mixing factor:0\u2212>35%learning algorithmfrozenDay 1Day 2Day 3Day 4Day 5Day 6Day 7Day 8days start with fullamplitudetargetsFreeze Freeze trials144 min.33 min.49 min.57 min.Success rateHand controlHand controlBMI mode\ftask (tried to decrease target size and the distance between targets) which had some effects of task\ndif\ufb01culty and on success rates. On days 8 and 9 we interleaved a BMI block with hand-control\nblocks. We saw that performance is better in hand-control than in BMI but not drastically so. In\nthe following sessions we discontinued all tweaking with the algorithm and we\u2019ve seen some steady\nimprovement in performance.\nWe repeated the freezing of model learning on two days (one of these session appears in \ufb01gure 3).\nIn all cases where we froze the model, we noticed that the monkey starts experiencing dif\ufb01culty in\ncontrolling the cursor after a period of 10-15 minutes and stopped working completely when this\nhappened. As stated earlier, in most BMI experiments the subjects interact with \ufb01xed models. One\npossible explanation for the deterioration is that because our monkey is trained in using an adaptive\nmodel it does not have experience in conforming to such a model\u2019s constraints. Instead, the opposite\nburden (that of following a drifting source of neural activity) falls on the algorithm\u2019s shoulders.\nAs mentioned in section 2, in BMI mode no hand movements are performed and therefore model\nlearning is based on our guess of what is the desired cursor trajectory (the monkey\u2019s emphintended\ncursor movement). We chose to design the desired trajectory as a time varying linear combination of\n\u02dcyi where\nthe cursor trajectory that the monkey saw and the target location: yi\n\u02dcyi is the target location on trial i. Note that this trajectory starts at the observed cursor location at\nthe trial start and ends at the target location (regardless of where the cursor actually was at the end\nof the trial).\n\n1 \u2212 t\ntfi\n\n\u02c6yi\nt + t\ntfi\n\n(cid:16)\n\nt =\n\n(cid:17)\n\n6 Summary\n\nThis study was motivated by the view that lifting overly simplifying assumptions and integrating\ndomain knowledge into machine learning algorithms can make signi\ufb01cant improvements to BMI\ntechnology. In turn, this technology can be used as a testbed for improved modeling of the interaction\nbetween the brain and environment, especially in visuomotor control. We showed in open-loop hand\ntracking experiments that the incorporation of a nonlinear movement model, interpreting the neural\nactivity as a whole (rather than as the sum of contributions made by single neurons) and allowing the\nmodel to adapt to changes, results in better predictions. The comparison was done against similar\nmodels that lack one or more of these properties. Finally, we showed that this model can be used\nsuccessfully in a real-time BMI setting, and that the added mathematical \u2019complications\u2019 result in a\nvery intuitive and high quality interface.\n\nReferences\n[1] D. M. Taylor, S. I. Helms, S.I. Tillery, and A.B. Schwartz. Direct cortical control of 3d neuroprosthetic devices. Science, 296(7):1829\u2013\n\n1832, 2002.\n\n[2] M.Velliste, S.Perel, M.C.Spalding, A.S. Whitford, and A. B. Schwartz. Cortical control of a prosthetic arm for self-feeding. Nature\n\n(online), May 2008.\n\n[3] S. Kakei, D. S. Hoffman, and P. L. Strick. Muscle and movement representations in the primary motor cortex. Science, 285:2136\u20132139,\n\n1999.\n\n[4] E. Vaadia, I. Haalman, M. Abeles, H. Bergman, Y. Prut, H. Slovin, and A. Aertsen. Dynamics of neuronal interactions in monkey cortex\n\nin relation to behavioral events. Nature, 373:515\u2013518, Febuary 1995.\n\n[5] W. Wu, Y. Gao, E. Bienenstock, J.P. Donoghue, and M.J. Black. Bayesian population coding of motor cortical activity using a kalman\n\n\ufb01lter. Neural Computation, 18:80\u2013118, 2005.\n\n[6] A. E. Brockwell, A. L. Rojas, and R. E. Kass. Recursive Bayesian Decoding of Motor Cortical Signals by Particle Filtering. J Neuro-\n\nphysiol, 91(4):1899\u20131907, 2004.\n\n[7] L. Shpigelman, Y. Singer, R. Paz, and E. Vaadia. Spikernels: Predicting hand movements by embedding population spike rates in\n\ninner-product spaces. Neural Computation, 17(3), 2005.\n\n[8] L. Shpigelman, K.Crammer, R. Paz, E.Vaadia, and Y.Singer. A temporal kernel-based model for tracking hand movements from neural\n\nactivities. In NIPS. 2005.\n\n[9] P.M.L. Drezet and R.F. Harrison. Support vector machines for system identi\ufb01cation. In UKACC, volume 1, pages 688\u2013692, Sep 1998.\n[10] M. Mart\u00ednez-Ram\u00f3n, J. Luis Rojo-\u00c1lvarez, G. Camps-Valls, J. Mu\u00f1oz-Mar\u00ed, A. Navia-V\u00e1zquez, E. Soria-Olivas, and A. R. Figueiras-\n\nVidal. Support vector machines for nonlinear kernel ARMA system identi\ufb01cation. IEEE Trans. Neural Net., 17(6):1617\u20131622, 2006.\n\n[11] G. E. P. Box and G. M. Jenkins. Time Series Analysis: Forecasting and Control. Prentice Hall PTR, Upper Saddle River, NJ, USA, 1994.\n[12] B. Schoelkopf and A. J. Smola. Learning with Kernels. The MIT Press, Cambridge, MA, 2002.\n[13] L.R.Hochberg, M.D. Serruya, G. M. Friehsand J.A. Mukand, M.Saleh, A. H. Caplan, A. Branner, D. Chen, R. D. Penn, and J. P.\n\nDonoghue. Neuronal ensemble control of prosthetic devices by a human with tetraplegia. Nature, 442:164\u2013171, 2006.\n\n8\n\n\f", "award": [], "sourceid": 786, "authors": [{"given_name": "Lavi", "family_name": "Shpigelman", "institution": null}, {"given_name": "Hagai", "family_name": "Lalazar", "institution": null}, {"given_name": "Eilon", "family_name": "Vaadia", "institution": null}]}