Keywords

1 Introduction

A cyborg is an organism with both biological and electronic parts. A variety of cyborgs or biobots were developed in recent years, such as rats, moths, cockroaches, and beetles [1,2,3,4,5]. Among these achievements, researchers had already been able to steer the animals or insects manually with the help of a computer or remote controller. Unlike previous work, we want to move one step forward by achieving the target of navigating a cyborg with human intention directly. To build a system that enables real-time control of a cyborg with human brain, at least two parts need to be set up. One part is a subsystem used to realize the functional control of a cyborg, and the other one is a subsystem used to translate the human brain signals. Fortunately, it’s possible to realize such two key parts with current technologies.

Neural electrical stimulation was widely used when developing cyborgs based on the increasing understanding of flight dynamics and the neurophysiology of animals or insects [6]. We aim at developing a kind of cyborg based on the cockroach, due to its robust performance and easy implementability. The cockroach antennas used for navigation during walking are important sensory organs that can generate multiple sensations (such as tactile, thermal, humidity and olfactory) [7]. When sending specific micro-electrical pulse trains through the antenna nerve, stimulation information will activate the descending mechanosensory interneurons (DMIs) (interneurons with the largest caliber axons descending to thoracic levels from the brain) and subsequently activate the thoracic motor centers, then evasive behaviors such as turning will be elicited [8,9,10]. Therefore, a neural interface used to control the walking direction of a cockroach with a micro electrical stimulator needed be developed in this work.

Brain-computer interfaces (BCIs) can help people communicate with the external world through measuring and translating brain activities without involving muscular movements or peripheral nervous system [11]. In this study, we choose the steady-state visual evoked potential (SSVEP) based BCI to recognize the human intention because SSVEP has high signal-to-noise ratio (SNR) and information transfer rate (ITR) and is currently safe, reliable, versatile and robust in the available BCIs including invasive and non-invasive BCIs [12,13,14,15].

Based on the SSVEP-based BCI and neural stimulation technology, we build up an all-chain wireless system that enables controlling the walking directions of a cockroach with human brain directly.

2 Methodology

BCI Implementation. The framework of the developed system is shown in Fig. 1. A three-state SSVEP-based BCI was used to decode the controller’s control intention. Three flashing square blocks represented the stimulation source of SSVEP [15, 16], which were located separately in the upper middle, lower left and lower right on a PC screen. The flickering frequency of each block was set as 12.5, 8.33, 6.818 Hz, denoting the rest, left-turn and right-turn control commands respectively. The human subject (controller) sat in front of the LCD screen of PC to manage direction control, wearing a portable EEG capture device (EPOC, Emotiv System Inc.) (Fig. 2a). Because the SSVEP mainly appears in the visual cortex, EEG signals from four channels (locations PO3, PO4, O1, O2 according to the international extended 10/20 system, with two CMS/DRL reference electrodes placed on C5/C6) were used for analysis. All electrode impedances were kept below 10 kΩ, and the sampling frequency was 128 Hz. A notch filter at 50 and 60 Hz was applied to process the EEG signals and the band pass is set to 0.16–43 Hz.

Fig. 1
figure 1

System framework. Four modules are included in the system: (1) the EEG acquisition module is used to obtain the brain signals from the human; (2) the host computer runs the SSVEP program and acts as an integrated platform; (3) data travels to the cyborg cockroach and its USB-based adapter; and (4) the wireless video capture module is used to transfer the real-time images of the cyborg to the LCD screen of PC for the human

Full size image
Fig. 2
figure 2

Experimental setup. a A subject (controller) wearing a portable BCI device performed the experiment to acquire the SSVEP signals. b A cyborg cockroach (receiver) was made successfully after surgery

Full size image

Cyborg Implementation. A cyborg cockroach acting as a receiver was developed after simple surgical process by the experimenters (Fig. 2b). The live Madagascar hissing cockroaches (Gromphadorhina portentosa) were adopted to make the cyborgs, since they are strong and large (about 50–80 mm) and have a slow walking speed. We surgically installed a microstimulator (Roboroach, v1.1b, Backyard Brains Inc. US) in the cockroach by inserting three tiny silver needles (left, right, ground electrode, 0.06 mm bare/0.08 mm coated) into the cockroach antennae (left, right) and the first segment of the thorax separately. The electrical stimulation pattern for the cyborg was a monopole square pulse with 1.5 V, 50 Hz, 50% duty cycle, and 500-ms pulse width. This configuration could generate a modest and proper reaction of the cyborgs, and therefore guaranteed the good online control performance.

Online Communication. Two custom software tools were written for the system. One was the SSVEP program, which included data acquisition, online analysis, and graphic user interface. The other software tool was used to realize the real-time communication between the SSVEP program and the microstimulator placed on the back of the cyborg.

Video Capture. We utilized a wireless video capture module in the SSVEP program to visually obtain the real-time response of the cyborg, and projected it to the LCD screen for the human subject (Fig. 4). Therefore, a closed-loop control system was established as shown in Fig. 1.

Experimental Paradigm. Three cyborg cockroaches were made, and three healthy human subjects were recruited. In each experiment, a cockroach was placed at a distance of about 1.5 m from the host computer. The human controller took online control of the cyborg to complete walking along the presetting tracks after the cyborg started to move forward from the start point of the track with a certain speed range (1.5–5.5 cm/s). Ten online-control trials were conducted for each subject and cyborg (Fig. 3). Before online control, each subject completed a 120-s training run first to optimize the classifier of SSVEP. A 120-s rest was given between two consecutive trials to minimize the effects of fatigue from both the humans and insects. In addition, experiments for control groups were also conducted in this study. Three cyborgs walked along the designed track freely without control from any human subject for ten trials separately in control groups. Two kinds of tracks were designed and tested for the system in the experiments: S-shape and obstacle-avoidance tracks.

Fig. 3
figure 3

Experimental paradigm. Controllers manage online control of different cyborgs with two different tracks (S-shape track and obstacle-avoidance track)

Full size image

The first track used in the online control experiments was an S-shape track (135 mm (W) * 750 mm (L)) (Fig. 4a). Completing walking along the S-shape track without going outside of the boundary was counted as a successful trial in the online control experiment.

Fig. 4
figure 4

Online control experiment. a One subject controlled a cyborg to walk along the S-shape track with brain signals. b One subject controlled a cyborg to walk along the obstacle-avoiding track with brain signals

Full size image

The other one was an obstacle-avoiding track (270 mm (W) * 750 mm (L)) (Fig. 4b). Controllers performed the online control of the cyborg to finish walking along the track from one side to the other side without crossing obstacles (red dots) on the sheet. Walking through the entire obstacle-avoiding track without touching any obstacles was counted as a successful trial.

Besides the above single-control experiments, we also conducted a double-control experiment to primarily explore the possibility of applying the designed system to further entertainment in daily life (Fig. 5). In this experiment, two human subjects each took control of a cyborg to complete walking along the S-shape track (the same size as in single-control paradigm) in the form of competition. Two cyborgs started moving forward in the same time, and between the two controllers, the one who navigated the cyborg to walk inside the track and reach the finish line first won the trial. Ten trials in total were completed in this experimental paradigm.

Fig. 5
figure 5

Double-control experiment. In this contest, two human subjects navigated two cyborgs to walk along the two S-shape tracks, respectively. The competition aimed to show which one was the first to reach the finish line

Full size image

3 Results

During the online control experiments, the cyborgs could produce quick responses to the applied invasive neural stimulation, and the time measured from sending a command from SSVEP to the completion of the cyborg’s reaction was about 772 ms in the present system. The cyborgs showed accurate response to the applied stimulation in most cases as well, and the mean response accuracy to the control commands reached 89.5 ± 15%. On the human side, average classification accuracy of SSVEP across three subjects in training sessions reached 86.0 ± 10.4%, indicating that both the BCI and neural stimulation subsystems in this study have the possibility to be used in an online control system.

The experimental results showed that all subjects could navigate all cyborgs to accomplish walking along both the different two tracks. The mean success rate for the online experiments achieved with this system was 20% for the S-shape track. Although not especially high, it was significantly higher than the value achieved in the control groups (0%) (t = 3.464, P = 0.0085). When using the obstacle-avoiding track, the success rate of online control could reach 40% (Fig. 4b). A demonstration video of a successful navigation of S-shape track is available through the link (https://www.youtube.com/watch?v=k5t6WkTkJkA).

An entertainment contest (double-control experiment) was also conducted, in which two persons competed to control two cockroaches, respectively. In this interesting paradigm, the success rate of navigation for each subject was the same as that in the single-control experiment (~20%). If both subjects were required to completely and successfully navigate in the same trial, this formidable task was still possible, but the average success rate was very low (~5%). Generally using the cyborgs that were more sensitive to the micro-stimulation would be more likely to win the contest.

4 Discussion

We presented a feasible method to navigate a cyborg with human brain in this study. The human subjects could successfully steer cockroaches to make desired turns with an SSVEP-based BCI and micro invasive neural stimulation. Using only 4 channels on the scalp of visual cortex with a portable EEG device, the SSVEP achieved high classification accuracy. However, misclassification appeared frequently in the SSVEP during the shifting period from one intention to another intention. Therefore, developing a program to detect transitions of the EEG signal to different frequencies before classification in further studies may help the SSVEP achieve more accurate classification and can be used in the asynchronous BCIs as well.

The performance of the online control varied among subjects and cyborgs. The biological factors of human and cockroaches affected the success rate of the present system, which actually cannot be easily solved by current technologies. Latif et al. [17] tried to steer a cockroach manually to walk along an S-shape line and finally achieved a 10% success rate. However, navigating a cyborg successfully with human brain signals in this study is much more challenging, which requires continuously high level of accuracy from both sides of “controller” (human) and “receiver” (cyborg), and calls for constantly quick response from both human intention recognition and cyborg reaction as well.

From another point of view, some other factors related to the online control task may influence the control performance as well, such as constraint level of the tracks used, BCI skills and experience from the human and so on. The result shows that the success rate of online navigation increases from 20 to 40% when switching the S-shape track to the obstacle-avoiding track that allows more freedom of control. It demonstrates well that the task and experimental paradigm can also affect the success rate.

All these factors account for the success rate, which is not high enough even though both the “controller” and the “receiver” have a relatively high accuracy of information translation for a single decision. The double-control experiment is very interesting, but this protocol, which requires two human subjects to control two cyborgs to hit the finish line in sequence, is a rather high standard in current BCI applications. It is more complicated than the single-control experiments, and we will go on exploring this issue in future study. To some extent, the performance of online control with current system is reasonable and instructive.

At present, we are developing a new type of wireless microstimulator with multiple stimulation channels and modes, which can technically contribute to the improvement of the control performance as well.

This work has realized the idea on utilizing the brain signals to steer a cyborg continuously for the first time. The idea may be used for detection in complex and dangerous environments in far future. Most importantly, this study also succeeds in building up an embryonic virtual brain-to-brain interface (BBI) to functionally transfer information from one brain to another [18,19,20]. With nogoing efforts from researchers, we believe that more modalities of both BCI and cyborg technologies will be developed and used in a variety of ways in the future.