Performance and Implementations of Vibrotactile Brain-Computer Interface with Ipsilateral and Bilateral Stimuli

SUN Hongyan ()， JIN Jing ( )， ZHANG Yu ( )， WANG Bei ( )， WANG Xingyu (行愚)

Key Laboratory of Advanced Control and Optmization for Chemical Processes， East China University of Science and Technology， Shanghai 200237， China

Abstract: The tactile P300 brain-computer interface (BCI) is related to the somatosensory perception and response of the human brain， and is different from visual or audio BCIs. Recently， several studies focused on the tactile stimuli delivered to different parts of the human body. Most of these stimuli were symmetrically bilateral. Only a few studies explored the influence of tactile stimuli laterality. In the current study， we extensively tested the performance of a vibrotactile BCI system using ipsilateral stimuli and bilateral stimuli. Two vibrotactile P300-based paradigms were tested. The target stimuli were located on the left and right forearms for the left forearm and right forearm (LFRF) paradigm, and on the left forearm and calf for the left forearm and left calf (LFLC) paradigm. Ten healthy subjects participated in this study. Our experiments and analysis showed that the bilateral paradigm (LFRF) elicited larger P300 amplitude and achieved significantly higher classification accuracy than the ipsilateral paradigm (LFLC). However， both paradigms achieved classification accuracies higher than 70% after the completion of several trials on average， which was usually regarded as the minimum accuracy level required for BCI system to be deemed useful.

Key words: brain-computer interface (BCI); tactile; P300; ipsilateral stimuli; bilateral stimuli paradigm; left forearm; right forearm; left calf

Introduction

Brain-computer interface (BCI) is a human-computer interaction technique， which is independent of normal peripheral nerves or muscle tissues[1-3]. The P300-based BCI system is often used in practice because of its high classification accuracy and information transfer rate[4-6]. The P300 signal can be evoked by visual， auditory， or tactile stimuli[5， 7-10]. The visual-based P300 BCI was first designed by Farwell and Donchin and was a 6×6 matrix speller system[4]. Hilletal. first explored the auditory-based P300 BCI by using a two-stimuli sequence: deviant and standard tones[9]. Moreover， the tactile-based P300 BCI was first proposed by Brouwer and van Erp and the system involved locating six vibrotactile stimulators located around the subject’s waist[10]. Some studies have compared the three stimuli modalities in P300 BCIs[11-12]. The tactile stimuli achieved the lowest accuracy compared with the other modalities for healthy users[11]; however， tactile stimuli obtained the best performance in patients with Amyotrophic Lateral Sclerosis (ALS) patients[12]. In addition， the tactile-based P300 BCI has been used to assess locked-in patients’ cognitive function and help them to communicate with the outside world[13-15]. Therefore， the tactile BCI system is a viable alternative to the existing BCI systems[12， 16-17].

In recent years， several tactile P300 BCI systems have been tested successfully， and researchers have tested vibrotactile stimulators on different parts of the human body. Van der Waaletal. introduced a tactile speller for communication by stimulating fingertips with short mechanical taps[18]. Morietal. delivered tactile stimuli to 10 fingers and 5 chest positions， and confirmed the feasibility of near-real-time operation of robotic vehicles[19-20]. Rutkowski and Mori designed a tactile and bone-conduction auditory BCI system by delivering vibrotactile stimuli to the head of participants[21]. Kodamaetal. demonstrated the effectiveness of BCI by delivering vibrotactile stimuli to the users’ back area[22]. Tactile event-related potential (ERP) was also used successfully to deliver navigation commands to control a wheelchair by placing tactile stimulators on the legs， abdomen， neckor back[23-24]. Furthermore， multisensory modalities-based BCI system have also been considered by combining tactile stimuli with visual stimuli or auditory stimuli， and bimodal stimuli was confirmed to boost BCI performance[25-27].

However, in previous studies， the vibrotactile was symmetrically bilateral， that is， stimulators were distributed on both sides of the body. Studies on the ipsilateral stimuli in tactile P300 BCIs are scarce. Kaufmannetal. studied ipsilateral tactile stimuli by placing four tactile stimulation units on a locked-in patient’s left arm[12]. The difference between ipsilateral tactile stimuli and bilateral tactile stimuli has not been thoroughly investigated. In reality， a large group of patients with unilateral somatosensory cortex injury often have somatosensory dysfunction， which can be caused by unilateral cerebral lesions or stroke[28-30]. The bilateral vibrated P300 BCI may not function well for these patients， and the ipsilateral stimuli may obtain better performance in this case compared with the bilateral stimuli. According to the somatosensory mechanism， the sensory information received from one side of the body mainly projects to the contralateral brain hemisphere[31]， and different body parts have different tactile acuity values[32-33]. Therefore， we hypothesized that the ipsilateral stimuli would have a different performance with the bilateral tactile stimuli， and would be advantageous in patients with unilateral somatosensory cortex injury. In this study， two paradigms that targeted the left forearm and right forearm (LFRF) paradigm and the left forearm and left calf (LFLC) paradigm were investigated and compared to show the performance differences between bilateral stimuli and ipsilateral stimuli.

1 Methods and Material

1.1 Subjects and data acquisition

A total of 10 healthy subjects (3 females and 7 males， aged 24-29) participated in this study， which were labelled as S1， S2， …， S10. The 10 participants signed a written consent form prior to this experiment and were paid 100 RMB for their participation. The local ethics committee approved the consent form and experimental procedures before any individuals participated. All participants had intact tactile sensation in their limbs.

Electroencephalograph(EEG) data were recorded by a g.USBamp and a g.EEGcap (Guger Technologies， Graz， Austria)， and band-pass filtered between 0.1 Hz and 30.0 Hz. The sampling frequency was set at 256.0 Hz. Eight electrodes (FCz， C3， Cz， C4， CP1， CPz， CP2， and Pz) were selected in accordance with the international 10-20 system. The right mastoid and front electrode (FPz) were separately used as the reference and the ground.

1.2 Stimuli and procedure

The mindBEAGLE system was used in this study (g.tec Medical Engineering GmbH， Schiedlberg， Austria)， which provided the stimulus presentation and data recording. This system was also used in some studies of consciousness assessment and communication for patients[13-15]. In this system， the vibrotactile stimuli were produced by g.VIBROstims powered by a g.STIMbox (g.tec Medical Engineering GmbH， Schiedlberg， Austria). Figure 1 shows a schematic indication of vibrotactile stimulators’ locations， which were referenced in the mindBEAGLE user manual. The circle stimulator， which was placed approximately 100 mm above the ankle on the right calf， mainly delivered standard stimuli (non-target stimuli)， which was never selected as a target. The star stimulators in both paradigms mainly delivered deviant stimuli (target stimuli).

Fig.1 Illustration of two stimuli paradigms: placement of vibrotactile stimulators

In the bilateral paradigm， the reason for selecting LFRF as target stimuli placements， rather than fingertips， was to avoid unnecessary finger shaking caused by vibrotactile stimuli and reduce the influence of handedness， as the left fingertips may have less tactile acuity than the right fingertips for the right-handed[34]. On the basis of the bilateral paradigm， researchers found that the ipsilateral stimuli are located on the LFLC， which allowed for easy discrimination between targets by making enough spacious distribution[24]. The standard stimuli located on the right calf evoked a large P300， which would help users discriminate from the target stimuli easily.

Each paradigm was performed for four trial blocks， and each block consisted of 15 trials (Fig. 2). In two of the four trial blocks， one of the stimulators was set as the target. The sequence of stimuli delivery to the target and non-target parts of the body was chosen randomly during each trial. The probability of deviant stimuli was one-eighth and that of the standard stimuli was six-eighths. The stimulus duration was 100 ms and the inter-stimulus interval was 200 ms.

Fig.2 Experimental procedure (the sequence of eight vibrations in each trial was random)

1.3 Feature extraction procedure and classification scheme

EEG data had a time window of 600 ms after the onset of stimulus and 100 ms segment before， which was used for baseline correction. Raw EEG signals were band-pass filtered from 0.5 Hz to 30.0 Hz by a third-order Butterworth filter. The data were also down sampled from 154.0 Hz to 22.0 Hz by selecting every seventh sample. Consequently， the size of the feature vector was 8×22 (8 channels， 22 samples).

To analyze the offline data， researchers used winsorizing to remove the electrooculogram interference signals. The 10th and 90th percentiles were computed as the extreme value， and the EEG data lying over the extreme value were replaced by the 10th percentile or the 90th percentile[35].

Among the several classification measures[36]， Bayesian linear discriminant analysis was chosen as the classifier because it can provide an excellent and consistent classification performance by avoiding over-fitting[35].

1.4 Data analysis

Bit rate is often used as an objective measure to compare the performance of different paradigms. According to Wolpawetal.[1]， raw bit rate (RBR) is defined as

(1)

wherePdenotes the classification accuracy，Ndenotes the number of targets for each trial， andTdenotes the completion time of the target selection task.

The statistical analysis method used in this paper was paired samplet-test. Before statistically comparing classification accuracy and RBR， data were statistically tested for normal distribution (one-sample Kolmogorov-Smirnov test). The significance level was set to 0.05.

2 Results

2.1 ERP patterns

The averaged ERPs of the target and non-target stimuli for the two paradigms are visualized in Fig. 3. Differences between the LFRF paradigm and the LFLC paradigm in relevant ERP components and regions are clearly shown; specifically， the P300 over left and right hemispheres.

Fig. 3 Mean ERPs amplitude of target and non-target at eight EEG electrodes：(a) FCz; (b) C3; (c) Cz; (d) C4; (e) CP1; (f) CPz; (g) CP2; (h)Pz

We calculated the peak values of N200 at FCz and P300 at Cz for each subject (Fig. 4). In Fig.4 Mean shows the average and the error bars represent the standard deviation. Not all participants could evoke ERPs effectively， such as S2 and S3 in Fig.4(b). A paired samplet-test was used to show the peak values difference between the two paradigms. The result showed that the LFLC paradigm evoked significantly lower P300 amplitude than the LFRF paradigm (t=-2.682，p=0.025). No significant differences were found on the N200 (t=-2.141，p=0.061); however， 7 out of 10 subjects achieved larger N200 in the LFRF paradigm.

Fig.4 Peak values of ERP at different electrodes: (a) N200 at FCz; (b) P300 at C

Notably， the ERP evoked by the LFLC paradigm at C3 electrode were not significant (Fig. 3). Consequently， the ERP of each paradigm at C3 and C4 for 10 subjects were compared (Figs. 5(a) and 5(c))， and the peak values of P300 (target amplitudes minus non-target amplitudes) at C3 and C4 across all subjects were also analyzed (Figs. 5(b) and 5(d)). The results of thet-test showed that the LFRF paradigm had no significant difference in peak points of P300 between the two hemispheres (t=-0.088，p=0.932). However， the difference was clearly significant in the LFLC paradigm (t=-3.396，p=0.008). Figure 6 shows the topographic map of P300 and N200 activity elicited by the LFRF paradigm and LFLC paradigm， which were peak values extracted from 150 ms to 250 ms for N200 component， and from 300 ms to 400 ms for P300 component.

2.2 Classification accuracy and bit rate

Classification accuracy and information transfer rate bit rate are important criteria for assessing the performance of the BCI system. Figure 7 shows the classification accuracy and RBRs of the two paradigms. The results presented are for four-fold cross-validation. The classification accuracy and RBRs of the LFRF paradigm were better than those of the LFLC paradigm (Fig. 7(a)). The result of thet-test showed that the single trial accuracy of the LFRF paradigm was significantly higher than that of the LFLC paradigm (Fig. 7(b)，t=2.529，p=0.032).

Given that the two paradigms evoked different ERP components， we calculated the contributions of the N200 (between 150 ms and 250 ms) and P300 (between 300 ms and 400 ms) activities to the classifier performance[7]. Figure 8 shows the contribution from the N200 (between 150 and 300 ms) and P300 (between 300 and 450 ms) on the BCI performance[37]. In most subjects， the P300 activity is the most important for classification accuracy. In addition， the N200 also contributes to the classification accuracy.

Fig.5 Difference between the contralateral and ipsilateral hemisphere: (a) averaged ERPs in the LFRF paradigm; (b) peak values of P300 in the LFRF paradigm; (c) averaged ERPs in the LFLC paradigm; (d) peak values of P300 in the LFLC paradigm

Fig.6 Topographic map of P300 and N200 activity averaged across 10 subjects: (a) LFRF paradigm; (b) LFLC paradigm

Fig.8 Contributions of N200 and P300 time windows to BCI classification performance across subjects: (a) LFRF paradigm; (b) LFLC paradigm

The single target classification accuracy was calculated to determine the main difference between targets in each paradigm (Fig. 9). The results of thet-test showed that the classification accuracy of the left and right forearms in the LFRF paradigm were not significantly different (Fig. 9(a)，t=1.317，p=0.22). In the LFLC paradigm， 6 out of the 10 subjects achieved higher accuracy on the left forearm and the rest achieved higher accuracy on the left calf (Fig. 9(b)). However， the difference between the two targets was not significant (t=-1.317，p=0.22).

3 Discussions

Fig.9 Single target classification accuracy across 10 subjects: (a) LFRF paradigm; (b) LFLC paradigm

This paper aimed to survey the influence of the tactile stimuli laterality， and two vibrotactile P300-based paradigms were investigated and compared. The LFRF paradigm was used to produce bilateral stimuli， and the LFLC paradigm was used to produce ipsilateral stimuli. The results showed that the LFRF paradigm could evoke larger P300 component and achieve significantly higher classification accuracy than the LFLC paradigm.

Usually， the tactile stimuli were delivered symmetrically to both sides of the human body. When the stimulators were distributed on only one side of the body， the ERP evoked on the contralateral hemisphere of brain wassignificantly larger than the ipsilateral one (Figs. 5(a) and 5(b)). In the LFRF paradigm， the ERP amplitude was nearly equal over the two hemispheres (Figs. 5(c) and 5(d)). According to the mechanism of somatosensory processing， the information received from one side of the body primarily projects to the contralateral brain hemisphere[31]. Given the overlap of four trial blocks， the ERP evoked by ipsilateral stimuli were strengthened on the contralateral hemisphere. By contrast， the ERP evoked by bilateral stimuli were nearly equal on the bilateral hemispheres. A similar phenomenon was also found in the study of steady-state somatosensory evoked potentials (SSSEP). Giabbiconietal. reported that SSSEP amplitude could increase significantly at the front-central region contralateral to the attended stimulation side[38].

Thurlingsetal. studied the ERP components in both the unimodal and bimodal visual-tactile based BCI systems. Their result showed that the P100 was only identified for visual modality and the N200 was only identified for the tactile modality[26]. In this paper， only one subject evoked the P100 component in the LFRF paradigm， which may be caused by this subject’s visual attention on the target forearm. Figure 4 shows that the LFRF paradigm can evoke significantly larger P300 than the LFLC paradigm. This phenomenon might indicate that the task of separating target stimuli from non-target and distracter stimuli is more difficult with the ipsilateral paradigm， as the amplitude of P300 generally decreases when task difficulty increases[39-40]. In addition， 7 out of 10 subjects achieved higher N200 amplitude in the LFRF paradigm. Selective attention can enhance the N200 amplitude in tactile BCI system[41].

In this paper， the classification accuracy was calculated by four-fold cross-validation. The bilateral paradigm achieved significantly higher classification accuracy than the ipsilateral paradigm (Fig. 7(b)). Different from the visual P300 BCI， neither of the two paradigms could achieve classification accuracy of up to 100% (Fig. 7(a)). This phenomenon is consistent with the previous studies on tactile P300 BCIs[10， 18， 24-27]. In addition， after the completion of several trials to obtain averages， both paradigms achieved a classification accuracy higher than 70% (Fig. 7(a))， which is the minimum accuracy percentage needed for a BCI system to be deemed useful[33].

In this study， the forearms and calves were selected as the vibrotactile stimulators’ placement， which allowed for easy discrimination between targets due to spacious distribution[24]. Some studies reported that they also located the stimulators at respective body positions (left thigh， right thigh， abdomen and low neck)， achieving higher BCI performance[23-24]. In Fig. 9， the statistical analysis of single target accuracy showed no significant difference between the two targets in each paradigm. Different parts of the body have different tactile acuity. The primary somatosensory cortex (S1) of each hemisphere contains a complete topographically organized representation of the contralateral side of the body， which is known as somatosensory “homunculus”[32， 42]. Weinstein reported that body parts extensively represented in S1 have better spatial tactile acuity[43]. In this paper， the forearm and calf have less tactile acuity compared with fingers[44]， and the difference between the two targets in each paradigm was small and not quite significant， as illustrated by the single target accuracy in Fig. 9.

4 Conclusions

This study presented two paradigms to survey the difference between LFLC stimuli and LFRF stimuli. The results showed that the LFRF paradigm (bilateral stimuli) could achieve significantly higher classification accuracy than the LFLC paradigm (ipsilateral stimuli). Future research should be performed to improve the current study by using online protocol and additional vibrotactile stimulators. Moreover， the two presented paradigms can be tested on patients with apoplectic hemiplegia.