{"title": "EEG-Based Brain-Computer Interaction: Improved Accuracy by Automatic Single-Trial Error Detection", "book": "Advances in Neural Information Processing Systems", "page_first": 441, "page_last": 448, "abstract": "Brain-computer interfaces (BCIs), as any other interaction modality based on physiological signals and body channels (e.g., muscular activity, speech and gestures), are prone to errors in the recognition of subject's intent. An elegant approach to improve the accuracy of BCIs consists in a verification procedure directly based on the presence of error-related potentials (ErrP) in the EEG recorded right after the occurrence of an error. Six healthy volunteer subjects with no prior BCI experience participated in a new human-robot interaction experiment where they were asked to mentally move a cursor towards a target that can be reached within a few steps using motor imagination. This experiment confirms the previously reported presence of a new kind of ErrP. These Interaction ErrP\" exhibit a first sharp negative peak followed by a positive peak and a second broader negative peak (~290, ~350 and ~470 ms after the feedback, respectively). But in order to exploit these ErrP we need to detect them in each single trial using a short window following the feedback associated to the response of the classifier embedded in the BCI. We have achieved an average recognition rate of correct and erroneous single trials of 81.8% and 76.2%, respectively. Furthermore, we have achieved an average recognition rate of the subject's intent while trying to mentally drive the cursor of 73.1%. These results show that it's possible to simultaneously extract useful information for mental control to operate a brain-actuated device as well as cognitive states such as error potentials to improve the quality of the brain-computer interaction. Finally, using a well-known inverse model (sLORETA), we show that the main focus of activity at the occurrence of the ErrP are, as expected, in the pre-supplementary motor area and in the anterior cingulate cortex.\"", "full_text": "EEG-Based Brain-Computer Interaction: Improved\nAccuracy by Automatic Single-Trial Error Detection\n\nPierre W. Ferrez\n\nIDIAP Research Institute\n\nCentre du Parc\n\nAv. des Pr\u00b4es-Beudin 20\n\n1920 Martigny, Switzerland\n\npierre.ferrez@idiap.ch\n\nJos\u00b4e del R. Mill\u00b4an\n\nIDIAP Research Institute\n\nCentre du Parc\n\nAv. des Pr\u00b4es-Beudin 20\n\n1920 Martigny, Switzerland\njose.millan@idiap.ch \u2217\n\nAbstract\n\nBrain-computer interfaces (BCIs), as any other interaction modality based on\nphysiological signals and body channels (e.g., muscular activity, speech and ges-\ntures), are prone to errors in the recognition of subject\u2019s intent. An elegant ap-\nproach to improve the accuracy of BCIs consists in a veri\ufb01cation procedure di-\nrectly based on the presence of error-related potentials (ErrP) in the EEG recorded\nright after the occurrence of an error. Six healthy volunteer subjects with no prior\nBCI experience participated in a new human-robot interaction experiment where\nthey were asked to mentally move a cursor towards a target that can be reached\nwithin a few steps using motor imagination. This experiment con\ufb01rms the previ-\nously reported presence of a new kind of ErrP. These \u201cInteraction ErrP\u201d exhibit a\n\ufb01rst sharp negative peak followed by a positive peak and a second broader negative\npeak (\u223c290, \u223c350 and \u223c470 ms after the feedback, respectively). But in order to\nexploit these ErrP we need to detect them in each single trial using a short win-\ndow following the feedback associated to the response of the classi\ufb01er embedded\nin the BCI. We have achieved an average recognition rate of correct and erroneous\nsingle trials of 81.8% and 76.2%, respectively. Furthermore, we have achieved an\naverage recognition rate of the subject\u2019s intent while trying to mentally drive the\ncursor of 73.1%. These results show that it\u2019s possible to simultaneously extract\nuseful information for mental control to operate a brain-actuated device as well\nas cognitive states such as error potentials to improve the quality of the brain-\ncomputer interaction. Finally, using a well-known inverse model (sLORETA), we\nshow that the main focus of activity at the occurrence of the ErrP are, as expected,\nin the pre-supplementary motor area and in the anterior cingulate cortex.\n\n1 Introduction\n\nPeople with severe motor disabilities (spinal cord injury (SCI), amyotrophic lateral sclerosis (ALS),\netc.) need alternative ways of communication and control for their everyday life. Over the past two\ndecades, numerous studies proposed electroencephalogram (EEG) activity for direct brain-computer\ninteraction [1]-[2]. EEG-based brain-computer interfaces (BCIs) provide disabled people with new\ntools for control and communication and are promising alternatives to invasive methods. However,\nas any other interaction modality based on physiological signals and body channels (e.g., muscular\nactivity, speech and gestures), BCIs are prone to errors in the recognition of subject\u2019s intent, and\nthose errors can be frequent. Indeed, even well-trained subjects rarely reach 100% of success. In\n\u2217This work is supported by the European IST Programme FET Project FP6-003758 and by the Swiss Na-\ntional Science Foundation NCCR \u201cIM2\u201d. This paper only re\ufb02ects the authors\u2019 views and funding agencies are\nnot liable for any use that may be made of the information contained herein.\n\n1\n\n\fcontrast to other interaction modalities, a unique feature of the \u201cbrain channel\u201d is that it conveys both\ninformation from which we can derive mental control commands to operate a brain-actuated device\nas well as information about cognitive states that are crucial for a purposeful interaction, all this on\nthe millisecond range. One of these states is the awareness of erroneous responses, which a number\nof groups have recently started to explore as a way to improve the performance of BCIs [3]-[6].\nIn particular, [6] recently reported the presence of a new kind of error potentials (ErrP) elicited by\nerroneous feedback provided by a BCI during the recognition of the subject\u2019s intent. In this study\nsubjects were asked to reach a target by sending repetitive manual commands to pass over several\nsteps. The system was executing commands with an 80% accuracy, so that at each step there was\na 20% probability that the system delivered an erroneous feedback. The main components of these\n\u201cInteraction ErrP\u201d are a negative peak 250 ms after the feedback, a positive peak 320 ms after the\nfeedback and a second broader negative peak 450 ms after the feedback. To exploit these ErrP for\nBCIs, it is mandatory to detect them no more in grand averages but in each single trial using a\nshort window following the feedback associated to the response of the BCI. The reported average\nrecognition rates of correct and erroneous single trials are 83.5% and 79.2%, respectively. These\nresults tend to show that ErrP could be a potential tool to improve the quality of the brain-computer\ninteraction. However, it is to note that in order to isolate the issue of the recognition of ErrP out of\nthe more dif\ufb01cult and general problem of a whole BCI where erroneous feedback can be due to non-\noptimal performance of both the interface (i.e., the classi\ufb01er embedded into the interface) and the\nuser himself, the subject delivered commands manually. The key issue now is to investigate whether\nsubjects also show ErrP while already engaged in tasks that require a high level of concentration\nsuch as motor imagination, and no more in easy tasks such as pressing a key.\nThe objective of the present study is to investigate the presence of these ErrP in a real BCI task.\nSubjects don\u2019t deliver manual commands anymore, but are focussing on motor imagination tasks\nto reach targets randomly selected by the system. In this paper we report new experimental results\nrecorded with six healthy volunteer subjects with no prior BCI experience during a simple human-\nrobot interaction that con\ufb01rm the previously reported existence of a new kind of ErrP [6], which\nis satisfactorily recognized in single trials using a short window just after the feedback. Further-\nmore, using a window just before the feedback, we report a 73.1% accuracy in the recognition of\nthe subject\u2019s intent during mental control of the BCI. This con\ufb01rms the fact that EEG conveys si-\nmultaneously information from which we can derive mental commands as well as information about\ncognitive states and shows that both can be suf\ufb01ciently well recognized in each single trials to pro-\nvide the subject with an improved brain-computer interaction. Finally, using a well-known inverse\nmodel called sLORETA [7] that non-invasively estimates the intracranial activity from scalp EEG,\nwe show that the main focus of activity at the occurrence of ErrP seems to be located in the pre-\nsupplementary motor area (pre-SMA) and in the anterior cingulate cortex (ACC), as expected [8][9].\n\nFigure 1: Illustration of the protocol. (1) The target (blue) appears 2 steps on the left side of the cursor (green).\n(2) The subject is imagining a movement of his/her left hand and the cursor moves 1 step to the left. (3) The\nsubject still focuses on his/her left hand, but the system moves the cursor in the wrong direction. (4) Correct\nmove to the left, compensating the error. (5) The cursor reaches the target. (6) A new target (red) appears 3\nsteps on the right side of the cursor, the subject will now imagine a movement of his/her right foot. The system\nmoved the cursor with an error rate of 20%; i.e., at each step, there was a 20% probability that the robot made\na movement in the wrong direction.\n\n2 Experimental setup\n\nThe \ufb01rst step to integrate ErrP detection in a BCI is to design a protocol where the subject is fo-\ncussing on a mental task for device control and on the feedback delivered by the BCI for ErrP\n\n2\n\n\fdetection. To test the ability of BCI users to concentrate simultaneously on a mental task and to be\naware of the BCI feedback at each single trial, we have simulated a human-robot interaction task\nwhere the subject has to bring the robot to targets 2 or 3 steps either to the left or to the right. This\nvirtual interaction is implemented by means of a green square cursor that can appear on any of 20\npositions along an horizontal line. The goal with this protocol is to bring the cursor to a target that\nrandomly appears either on the left (blue square) or on the right(red square) of the cursor. The target\nis no further away than 3 positions from the cursor (symbolizing the current position of the robot).\nThis prevents the subject from habituation to one of the stimuli since the cursor reaches the target\nwithin a small number of steps. Figure 1 illustrates the protocol with the target (blue) initially po-\nsitioned 2 steps away on the left side of the cursor (green). An error occurred at step 3) so that the\ncursor reaches the target in 5 steps. Each target corresponds to a speci\ufb01c mental task. The subjects\nwere asked to imagine a movement of their left hand for the left target and to imagine a movement\nof their right foot for the right target (note that subject n\u25e61 selected left foot for the left target and\nright hand for the right target). However, since the subjects had no prior BCI experience, the system\nwas not moving the cursor following the mental commands of the subject, but with an error rate of\n20%, to avoid random or totally biased behavior of the cursor.\nSix healthy volunteer subjects with no prior BCI experience participated in these experiments. After\nthe presentation of the target, the subject focuses on the corresponding mental task until the cursor\nreached the target. The system moved the cursor with an error rate of 20%; i.e., at each step, there\nwas a 20% probability that the cursor moved in the opposite direction. When the cursor reached a\ntarget, it brie\ufb02y turned from green to light green and then a new target was randomly selected by the\nsystem. If the cursor didn\u2019t reach the target after 10 steps, a new target was selected. As shown in\n\ufb01gure 2, while the subject focuses on a speci\ufb01c mental task, the system delivers a feedback about ev-\nery 2 seconds. This provides a window just before the feedback for BCI classi\ufb01cation and a window\njust after the feedback for ErrP detection for every single trial. Subjects performed 10 sessions of 3\nminutes on 2 different days (the delay between the two days of measurements varied from 1 week\nto 1 month), corresponding to \u223c75 single trials per session. The 20 sessions were split into 4 groups\nof 5, so that classi\ufb01ers were built using a group and tested on the following group. The classi\ufb01cation\nrates presented in Section 3 are therefore the average of 3 prediction performances: classi\ufb01cation of\ngroup n + 1 using group n to build a classi\ufb01er. This rule applies for both mental tasks classi\ufb01cation\nand ErrP detection.\n\nFigure 2: Timing of the protocol. The system delivers a feedback about every 2 seconds, this provides a\nwindow just before the feedback for BCI classi\ufb01cation and a window just after the feedback for ErrP detection\nfor every single trial. As a new target is presented, the subject focuses on the corresponding mental task until\nthe target is reached.\n\nEEG potentials were acquired with a portable system (Biosemi ActiveTwo) by means of a cap with\n64 integrated electrodes covering the whole scalp uniformly. The sampling rate was 512 Hz and\nsignals were measured at full DC. Raw EEG potentials were \ufb01rst spatially \ufb01ltered by subtracting\nfrom each electrode the average potential (over the 64 channels) at each time step. The aim of this\nre-referencing procedure is to suppress the average brain activity, which can be seen as underlying\nbackground activity, so as to keep the information coming from local sources below each electrode.\nThen for off-line mental tasks classi\ufb01cation, the power spectrum density (PSD) of EEG channels\nwas estimated over a window of one second just before the feedback. PSD was estimated using\nthe Welch method resulting in spectra with a 2 Hz resolution from 6 to 44 Hz. The most relevant\nEEG channels and frequencies were selected by a simple feature selection algorithm based on the\noverlap of the distributions of the different classes. For off-line ErrP detection, we applied a 1-10\n\n3\n\n\fHz bandpass \ufb01lter as ErrP are known to be a relatively slow cortical potential. EEG signals were\nthen subsampled from 512 Hz to 64 Hz (i.e., we took one point out of 8) before classi\ufb01cation,\nwhich was entirely based on temporal features. Indeed the actual input vector for the statistical\nclassi\ufb01er described below is a 150 ms window starting 250 ms after the feedback for channels FCz\nand Cz. The choice of these channels follows the fact that ErrP are characterized by a fronto-central\ndistribution along the midline.\nFor both mental tasks and ErrP classi\ufb01cation, the two different classes (left or right for mental tasks\nand error or correct for ErrP) are recognized by a Gaussian classi\ufb01er. The output of the statistical\nclassi\ufb01er is an estimation of the posterior class probability distribution for a single trial; i.e., the\nprobability that a given single trial belongs to one of the two classes. In this statistical classi\ufb01er,\nevery Gaussian unit represents a prototype of one of the classes to be recognized, and we use several\nprototypes per class. During learning, the centers of the classes of the Gaussian units are pulled\ntowards the trials of the class they represent and pushed away from the trials of the other class. No\nartifact rejection algorithm (for removing or \ufb01ltering out eye or muscular movements) was applied\nand all trials were kept for analysis. It is worth noting, however, that after a visual a-posteriori check\nof the trials we found no evidence of muscular artifacts that could have contaminated one condition\ndifferently from the other. More details on the Gaussian classi\ufb01er and the analysis procedure to rule\nout ocular/muscular artifacts as the relevant signals for both classi\ufb01ers (BCI itself and ErrP) can be\nfound in [10].\n\nFigure 3: (Top) Discriminant power (DP) of frequencies. Sensory motor rhythm (12-16 Hz) and some beta\n(Bottom) Discriminant power (DP) of electrodes. The most\ncomponents are discriminant for all subjects.\nrelevant electrodes are in the central area (C3, C4 and Cz) according to the ERD/ERD location for hand and\nfoot movement or imagination.\n\n3 Experimental results\n\n3.1 Mental tasks classi\ufb01cation\n\nSubject were asked to imagine a movement of their left hand when the left target was proposed and\nto imagine a movement of their right foot when the right target was proposed (note that subject n\u25e61\n\n4\n\n\fwas imagining left foot for the left target and right hand for the right target). The most relevant\nEEG channels and frequencies were selected by a simple feature selection algorithm based on the\noverlap of the distributions of the different classes. Figure 3 shows the discriminant power (DP) of\nfrequencies (top) and electrodes (bottom) for the 6 subject. For frequencies, the DP is based on the\nbest electrode, and for electrodes it is based on the best frequency. Table 1 shows the classi\ufb01cation\nrates for the two mental tasks and the general BCI accuracy for the 6 subjects and the average of\nthem, it also shows the features (electrodes and frequencies) used for classi\ufb01cation.\nFor all 6 subjects, the 12-16 Hz band (sensory motor rhythm (SMR)) appears to be relevant for\nclassi\ufb01cation. Subject 1, 3 and 5 show a peak in DP for frequencies around 25 Hz (beta band). For\nsubject 2 this peak in the beta band is centered at 20 Hz and for subject 6 it is centered at 30 Hz.\nFinally subject 4 shows no particular discriminant power in the beta band. Previous studies con\ufb01rm\nthese results. Indeed, SMR and beta rhythm over left and/or right sensorimotor cortex have been suc-\ncessfully used for BCI control [11]. Event-related de-synchronization (ERD) and synchronization\n(ERS) refer to large-scale changes in neural processing. During periods of inactivity, brain areas are\nin a kind of idling state with large populations of neurons \ufb01ring in synchrony resulting in an increase\nof amplitude of speci\ufb01c alpha (8-12 Hz) and beta (12-26 Hz) bands. During activity, populations of\nneurons work at their own pace and the power of this idling state is reduced, the cortex has become\nde-synchronized. [12]. In our case, the most relevant electrodes for all subjects are in the C3, C4 or\nCz area. These locations con\ufb01rm previous studies since C3 and C4 areas usually show ERD/ERS\nduring hands movement or imagination whereas foot movement or imagination are focused in the\nCz area [12].\n\nTable 1: Percentages (mean and standard deviations) of correctly recognized single trials for the 2 motor\nimagination tasks for the 6 subjects and the average of them. All subjects show classi\ufb01cation rates of about\n70-75% for motor imagination and the general BCI accuracy is 73%. Features used for classi\ufb01cation are also\nshown.\n\nElectrodes\n\nFrequencies\n\n[Hz]\n\n10 12 14 26\n\n10 12 14 18 20 22\n\n14 16 26\n\n12 14\n\n12 24 26\n\nCPz Cz CP6 CP4\n\n12 14 28 30 32\n\nC3 CP3 CP1 CPz CP2\n\nC4 CP4 P4\n\nC3 C4 C6 CP6 CP4\n\nCz C2 C4\nCz C4 CP4\n\n# 1*\n# 2\n# 3\n# 4\n# 5\n# 6\nAvg\n\n[%]\n\nLeft hand\n77.2 \u00b1 3.7\n71.8 \u00b1 9.0\n76.4 \u00b1 5.8\n79.6 \u00b1 1.6\n73.5 \u00b1 16.1\n77.9 \u00b1 7.4\n76.1 \u00b1 2.9\n\n[%]\n\nRight foot\n70.4 \u00b1 3.2\n80.9 \u00b1 7.1\n62.6 \u00b1 6.7\n66.3 \u00b1 10.1\n71.9 \u00b1 13.3\n69.0 \u00b1 13.7\n70.2 \u00b1 6.2\n\n[%]\n\nAccuracy\n73.8 \u00b1 4.8\n76.4 \u00b1 6.4\n69.5 \u00b1 9.8\n73.0 \u00b1 9.4\n72.7 \u00b1 1.1\n73.5 \u00b1 6.3\n73.1 \u00b1 4.2\n\n* Left foot and Right hand\n\nAll 6 subjects show classi\ufb01cation rates of about 70-75% for motor imagination. These \ufb01gures were\nachieved with a relatively low number of features (up to 5 electrodes and up to 6 frequencies) and the\ngeneral BCI accuracy is 73%. This level of performance can appear relatively low for a 2-class BCI.\nHowever, keeping in mind that \ufb01rst all subjects had no prior BCI experience and second that these\n\ufb01gures were obtained exclusively in prediction (i.e. classi\ufb01ers were always tested on new data), the\nperformance is satisfactory.\n\n3.2 Error-related potentials\n\nFigure 4 shows the averages of error trials (red curve), of correct trials (green curve) and the dif-\nference error-minus-correct (blue curve) for channel FCz for the six subjects (top). A \ufb01rst small\npositive peak shows up about \u223c230 ms after the feedback (t=0). A negative peak clearly appears\n\u223c290 ms after the feedback for 5 subjects. This negative peak is followed by a positive peak \u223c350\nms after the feedback. Finally a second broader negative peak occurs about \u223c470 ms after the\nfeedback. Figure 4 also shows the scalp potentials topographies (right) for the average of the six\nsubjects, at the occurrence of the four previously described peaks: a \ufb01rst fronto-central positivity\nappears after \u223c230 ms, followed by a fronto-central negativity at \u223c290 ms, a fronto-central positiv-\nity at \u223c350 ms and a fronto-central negativity at \u223c470 ms. All six subjects show similar ErrP time\ncourses whose amplitudes slightly differ from one subject to the other. These experiments seem to\ncon\ufb01rm the existence of a new kind of error-related potentials [6]. Furthermore, the fronto-central\n\n5\n\n\ffocus at the occurrence of the different peaks tends to con\ufb01rm the hypothesis that ErrP are generated\nin a deep brain region called anterior cingulate cortex [8][9] (see also Section 3.3).\nTable 2 reports the recognition rates (mean and standard deviations) for the six subjects plus the\naverage of them. These results show that single-trial recognition of erroneous and correct responses\nare above 75% and 80%, respectively. Beside the crucial importance to integrate ErrP in the BCI\nin a way that the subject still feels comfortable, for example by reducing as much as possible the\nrejection of actually correct commands, a key point for the exploitation of the automatic recognition\nof interaction errors is that they translate into an actual improvement of the performance of the BCI.\nTable 2 also show the performance of the BCI in terms of bit rate (bits per trial) when detection\nof ErrP is used or not and the induced increase of performance (for details see [6]). The bene\ufb01t of\nintegrating ErrP detection is obvious since it at least doubles the bit rate for \ufb01ve of the six subjects\nand the average increase is 124%.\n\nFigure 4: (Top) Averages of error trials (red curve), of correct trials (green curve) and the difference error-\nminus-correct (blue curve) for channel FCz for the six subjects. All six subjects show similar ErrP time courses\nwhose amplitudes slightly differ from one subject to the other. (Bottom) Scalp potentials topographies for the\naverage of the six subjects, at the occurrence of the four described peaks. All focuses are located in fronto-\ncentral areas, over the anterior cingulate cortex (ACC).\n\nTable 2: Percentages (mean and standard deviations) of correctly recognized error trials and correct trials for\nthe six subjects and the average of them. Table also show the BCI performance in terms of bit rate and its\nincrease using ErrP detection. Classi\ufb01cation rates are above 75% and 80% for error trials and correct trials,\nrespectively. The bene\ufb01t of integrating ErrP detection is obvious since it at least doubles the bit rate for \ufb01ve of\nthe six subjects.\n\nError\n[%]\n\n77.7 \u00b1 13.9\n75.4 \u00b1 5.5\n74.0 \u00b1 12.9\n84.3 \u00b1 7.7\n75.3 \u00b1 6.0\n70.7 \u00b1 11.4\n76.2 \u00b1 4.6\n\n[%]\n\nCorrect\n76.8 \u00b1 5.4\n80.1 \u00b1 7.9\n85.9 \u00b1 1.6\n80.1 \u00b1 5.5\n85.6 \u00b1 5.2\n82.2 \u00b1 5.1\n81.8 \u00b1 3.5\n\nBCI accuracy [%]\n\n(from Table 1)\n73.8 \u00b1 4.8\n76.4 \u00b1 6.4\n69.5 \u00b1 9.8\n73.0 \u00b1 9.4\n72.7 \u00b1 1.1\n73.5 \u00b1 6.3\n73.1 \u00b1 4.2\n\nBit rate [bits/trial]\n(ErrP)\n(no ErrP)\n0.345\n0.385\n0.324\n0.403\n0.371\n0.333\n0.359\n\n0.170\n0.212\n0.113\n0.159\n0.154\n0.166\n0.160\n\n# 1\n# 2\n# 3\n# 4\n# 5\n# 6\nAvg\n\nIncrease\n\n[%]\n103\n82\n187\n154\n141\n101\n124\n\n3.3 Estimation of intracranial activity\n\nEstimating the neuronal sources that generate a given potential map at the scalp surface (EEG)\nrequires the solution of the so-called inverse problem. This inverse problem is always initially\nundetermined, i.e.\nthere is no unique solution since a given potential map at the surface can be\n\n6\n\n\fgenerated by many different intracranial activity map. The inverse problem requires supplementary\na priori constraints in order to be univocally solved. The ultimate goal is to unmix the signals\nmeasured at the scalp and to attribute to each brain area its own estimated temporal activity. The\nsLORETA inverse model [7] is a standardized low resolution brain electromagnetic tomography.\nThis software, known for its zero localization error, was used as a localization tool to estimate\nthe focus of intracranial activity at the occurrence of the four ErrP peaks described in Section 3.2.\nFigure 5 shows Talairach slices of localized activity for the grand average of the six subjects at the\noccurrence of the four described peaks and at the occurrence of a late positive component showing\nup 650 ms after the feedback. As expected, the areas involved in error processing, namely the\npre-supplementary motor area (pre-SMA, Brodmann area 6) and the rostral cingulate zone (RCZ,\nBrodmann areas 24 & 32) are systematically activated [8][9]. For the second positive peak (350\nms) and mainly for the late positive component (650 ms), parietal areas are also activated. These\nassociative areas (somatosensory association cortex, Brodmann areas 5 & 7) could be related to\nthe fact that the subject becomes aware of the error. It has been proposed that the positive peak\nwas associated with conscious error recognition in case of error potentials elicited in reaction task\nparadigm [13]. In our case, activation of parietal areas after 350 ms after the feedback agrees with\nthis hypothesis.\n\nFigure 5: Talairach slices of localized activity for the grand average of the six subjects at the occurrence of\nthe four peaks described in Section 3.2 and at the occurrence of a late positive component showing up 650\nms after the feedback. Supplementary motor cortex and anterior cingulate cortex are systematically activated.\nFurthermore, for the second positive peak (350 ms) and mainly for the late positive component (650 ms),\nparietal areas are also activated. This parietal activation could re\ufb02ect the fact that the subject is aware of the\nerror.\n\n4 Discussion\n\nIn this study we have reported results on the detection of the neural correlate of error awareness for\nimproving the performance and reliability of BCI. In particular, we have con\ufb01rmed the existence of\na new kind of error-related potential elicited in reaction to an erroneous recognition of the subject\u2019s\nintention. More importantly, we have shown the feasibility of simultaneously and satisfactorily de-\ntecting erroneous responses of the interface and classifying motor imagination for device control at\nthe level of single trials. However, the introduction of an automatic response rejection strongly inter-\nferes with the BCI. The user needs to process additional information which induces higher workload\nand may considerably slow down the interaction. These issues have to be investigated when running\nonline BCI experiments integrating automatic error detection. Given the promising results obtained\nin this simulated human-robot interaction, we are currently working in the actual integration of on-\nline ErrP detection into our BCI system. The preliminary results are very promising and con\ufb01rm\nthat the online detection of errors is a tool of great bene\ufb01t, especially for subjects with no prior\nBCI experience or showing low BCI performance. In parallel, we are exploring how to increase the\nrecognition rate of single-trial erroneous and correct responses.\n\n7\n\n\fIn this study we have also shown that, as expected, typical cortical areas involved in error process-\ning such as pre-supplementary motor area and anterior cingulate cortex are systematically activated\nat the occurrence of the different peaks. The software used for the estimation of the intracranial\nactivity (sLORETA) is only a localization tool. However, Babiloni et al. [14] have recently devel-\noped the so-called CCD (\u201ccortical current density\u201d) inverse model that estimates the activity of the\ncortical mantle. Since ErrP seems to be generated by cortical areas, we plan to use this method to\nbest discriminate erroneous and correct responses of the interface. As a matter of fact, a key issue to\nimprove classi\ufb01cation is the selection of the most relevant current dipoles out of a few thousands. In\nfact, the very preliminary results using the CCD inverse model con\ufb01rm the reported localization in\nthe pre-supplementary motor area and in the anterior cingulate cortex and thus we may well expect\na signi\ufb01cant improvement in recognition rates by focusing on the dipoles estimated in those speci\ufb01c\nbrain areas.\nMore generally, the work described here suggests that it could be possible to recognize in real time\nhigh-level cognitive and emotional states from EEG (as opposed, and in addition, to motor com-\nmands) such as alarm, fatigue, frustration, confusion, or attention that are crucial for an effective and\npurposeful interaction. Indeed, the rapid recognition of these states will lead to truly adaptive in-\nterfaces that customize dynamically in response to changes of the cognitive and emotional/affective\nstates of the user.\n\nReferences\n[1] J.R. Wolpaw, N. Birbaumer, D.J. McFarland, G. Pfurtscheller, and T.M. Vaughan. Brain-computer inter-\n\nfaces for communication and control. Clinical Neurophysiology, 113:767\u2013791, 2002.\n\n[2] J. del R. Mill\u00b4an, F. Renkens, J. Mouri\u02dcno, and W. Gerstner. Non-invasive brain-actuated control of a mobile\n\nrobot by human EEG. IEEE Transactions on Biomedical Engineering, 51:1026\u20131033, 2004.\n\n[3] G. Schalk, J.R. Wolpaw, D.J. McFarland, and G. Pfurtscheller. EEG-based communication: presence of\n\nand error potential. Clinical Neurophysiology, 111:2138\u20132144, 2000.\n\n[4] B. Blankertz, G. Dornhege, C. Sch\u00a8afer, R. Krepki, J. Kohlmorgen, K.-R. M\u00a8uller, V. Kunzmann, F. Losch,\nand G. Curio. Boosting bit rates and error detection for the classi\ufb01cation of fast-paced motor commands\nbased on single-trial EEG analysis. IEEE Transactions on Neural Systems and Rehabilitation Engineer-\ning, 11(2):127\u2013131, 2003.\n\n[5] L.C. Parra, C.D. Spence, A.D. Gerson, and P. Sajda. Response error correction\u2014a demonstration of\nimproved human-machine performance using real-time EEG monitoring. IEEE Transactions on Neural\nSystems and Rehabilitation Engineering, 11(2):173\u2013177, 2003.\n\n[6] P.W. Ferrez and J. del R. Mill\u00b4an. You are wrong!\u2014Automatic detection of interaction errors from brain\n\nwaves. In Proc. 19th Int. Joint Conf. Arti\ufb01cial Intelligence, 2005.\n\n[7] R.D. Pascual-Marqui. Standardized low resolution brain electromagnetic tomography (sLORETA): Tech-\n\nnical details. Methods & Findings in Experimental & Clinical Pharmacology, 24D:5\u201312, 2002.\n\n[8] C.B. Holroyd and M.G.H. Coles. The neural basis of human error processing: Reinforcement learning,\n\ndopamine and the error-related negativity. Psychological Review, 109:679\u2013709, 2002.\n\n[9] K. Fiehler, M. Ullsperger, and Y. von Cramon. Neural correlates of error detection and error correction:\nIs there a common neuroanatomical substrate? European Journal of Neuroscience, 19:3081\u20133087, 2004.\n[10] P.W. Ferrez and J. del R. Mill\u00b4an. Error-related EEG potentials in brain-computer interfaces. In G. Dorn-\nhege, J. del R. Mill\u00b4an, T. Hinterberger, D. McFarland, and K.-R. M\u00a8uller, editors, Toward Brain-Computing\nInterfacing, pages 291\u2013301. The MIT Press, 2007.\n\n[11] D. McFarland and J.R. Wolpow. Sensorimotor rhythm-based brain-computer interface (BCI): Feature\nselection by regression improves performance. IEEE Transactions on Neural Systems and Rehabilitation\nEngineering, 13(3):372\u2013379, 2005.\n\n[12] G. Pfurtscheller and F.H. Lopes da Silva. Event-related EEG/MEG synchronization and desynchroniza-\n\ntion: Basic principles. Clinical Neurophysiology, 110:1842\u20131857, 1999.\n\n[13] S. Nieuwenhuis, K.R. Ridderinkhof, J. Blom, G.P.H. Band, and A. Kok. Error-related brain potentials are\ndifferently related to awareness of response errors: Evidence from an antisaccade task. Psychophysiology,\n38:752\u2013760, 2001.\n\n[14] F. Babiloni, C. Babiloni, L. Locche, F. Cincotti, P.M. Rossini, and F. Carducci. High-resolution electro-\nencephalogram: Source estimates of laplacian-transformed somatosensory-evoked potentials using realis-\ntic subject head model constructed from magnetic resonance imaging. Medical & Biological Engineering\nand Computing, 38:512\u2013519, 2000.\n\n8\n\n\f", "award": [], "sourceid": 103, "authors": [{"given_name": "Pierre", "family_name": "Ferrez", "institution": null}, {"given_name": "Jos\u00e9", "family_name": "Mill\u00e1n", "institution": null}]}