Abstract
A kind of cyborg was developed by surgically linking a portable microstimulator with the nerves of antennas of a live cockroach. Through applying specific micro electrical stimulation, the cyborg could be remotely controlled to make left and right turns. The motion intention could be retrieved from the human brain via brain-computer interface (BCI). Steady-state visual evoked potential (SSVEP) based-electroencephalography (EEG), as a robust BCI, was used to translate human intention. By merging the technologies of cyborg and BCI, it was possible to guide a live cockroach with human brain. Experiments with different paradigms were designed and conducted to verify the performance of the proposed system. The experimental results showed that the average success rates of both human BCI and cyborg reactions in a single decision were over 85%. The cyborg could be steered successfully via the human brain to complete walking along pre-set tracks with a 20% success rate.
Access provided by CONRICYT-eBooks. Download chapter PDF
Similar content being viewed by others
Keywords
- Brain-computer interface
- Electroencephalography
- Steady-state visual evoked potential
- Cyborg navigation
- Vockroach
1 Introduction
A cyborg is an organism with both biological and electronic parts. A variety of cyborgs or biobots were developed in recent years, such as rats, moths, cockroaches, and beetles [1,2,3,4,5]. Among these achievements, researchers had already been able to steer the animals or insects manually with the help of a computer or remote controller. Unlike previous work, we want to move one step forward by achieving the target of navigating a cyborg with human intention directly. To build a system that enables real-time control of a cyborg with human brain, at least two parts need to be set up. One part is a subsystem used to realize the functional control of a cyborg, and the other one is a subsystem used to translate the human brain signals. Fortunately, it’s possible to realize such two key parts with current technologies.
Neural electrical stimulation was widely used when developing cyborgs based on the increasing understanding of flight dynamics and the neurophysiology of animals or insects [6]. We aim at developing a kind of cyborg based on the cockroach, due to its robust performance and easy implementability. The cockroach antennas used for navigation during walking are important sensory organs that can generate multiple sensations (such as tactile, thermal, humidity and olfactory) [7]. When sending specific micro-electrical pulse trains through the antenna nerve, stimulation information will activate the descending mechanosensory interneurons (DMIs) (interneurons with the largest caliber axons descending to thoracic levels from the brain) and subsequently activate the thoracic motor centers, then evasive behaviors such as turning will be elicited [8,9,10]. Therefore, a neural interface used to control the walking direction of a cockroach with a micro electrical stimulator needed be developed in this work.
Brain-computer interfaces (BCIs) can help people communicate with the external world through measuring and translating brain activities without involving muscular movements or peripheral nervous system [11]. In this study, we choose the steady-state visual evoked potential (SSVEP) based BCI to recognize the human intention because SSVEP has high signal-to-noise ratio (SNR) and information transfer rate (ITR) and is currently safe, reliable, versatile and robust in the available BCIs including invasive and non-invasive BCIs [12,13,14,15].
Based on the SSVEP-based BCI and neural stimulation technology, we build up an all-chain wireless system that enables controlling the walking directions of a cockroach with human brain directly.
2 Methodology
BCI Implementation. The framework of the developed system is shown in Fig. 1. A three-state SSVEP-based BCI was used to decode the controller’s control intention. Three flashing square blocks represented the stimulation source of SSVEP [15, 16], which were located separately in the upper middle, lower left and lower right on a PC screen. The flickering frequency of each block was set as 12.5, 8.33, 6.818 Hz, denoting the rest, left-turn and right-turn control commands respectively. The human subject (controller) sat in front of the LCD screen of PC to manage direction control, wearing a portable EEG capture device (EPOC, Emotiv System Inc.) (Fig. 2a). Because the SSVEP mainly appears in the visual cortex, EEG signals from four channels (locations PO3, PO4, O1, O2 according to the international extended 10/20 system, with two CMS/DRL reference electrodes placed on C5/C6) were used for analysis. All electrode impedances were kept below 10 kΩ, and the sampling frequency was 128 Hz. A notch filter at 50 and 60 Hz was applied to process the EEG signals and the band pass is set to 0.16–43 Hz.
Cyborg Implementation. A cyborg cockroach acting as a receiver was developed after simple surgical process by the experimenters (Fig. 2b). The live Madagascar hissing cockroaches (Gromphadorhina portentosa) were adopted to make the cyborgs, since they are strong and large (about 50–80 mm) and have a slow walking speed. We surgically installed a microstimulator (Roboroach, v1.1b, Backyard Brains Inc. US) in the cockroach by inserting three tiny silver needles (left, right, ground electrode, 0.06 mm bare/0.08 mm coated) into the cockroach antennae (left, right) and the first segment of the thorax separately. The electrical stimulation pattern for the cyborg was a monopole square pulse with 1.5 V, 50 Hz, 50% duty cycle, and 500-ms pulse width. This configuration could generate a modest and proper reaction of the cyborgs, and therefore guaranteed the good online control performance.
Online Communication. Two custom software tools were written for the system. One was the SSVEP program, which included data acquisition, online analysis, and graphic user interface. The other software tool was used to realize the real-time communication between the SSVEP program and the microstimulator placed on the back of the cyborg.
Video Capture. We utilized a wireless video capture module in the SSVEP program to visually obtain the real-time response of the cyborg, and projected it to the LCD screen for the human subject (Fig. 4). Therefore, a closed-loop control system was established as shown in Fig. 1.
Experimental Paradigm. Three cyborg cockroaches were made, and three healthy human subjects were recruited. In each experiment, a cockroach was placed at a distance of about 1.5 m from the host computer. The human controller took online control of the cyborg to complete walking along the presetting tracks after the cyborg started to move forward from the start point of the track with a certain speed range (1.5–5.5 cm/s). Ten online-control trials were conducted for each subject and cyborg (Fig. 3). Before online control, each subject completed a 120-s training run first to optimize the classifier of SSVEP. A 120-s rest was given between two consecutive trials to minimize the effects of fatigue from both the humans and insects. In addition, experiments for control groups were also conducted in this study. Three cyborgs walked along the designed track freely without control from any human subject for ten trials separately in control groups. Two kinds of tracks were designed and tested for the system in the experiments: S-shape and obstacle-avoidance tracks.
The first track used in the online control experiments was an S-shape track (135 mm (W) * 750 mm (L)) (Fig. 4a). Completing walking along the S-shape track without going outside of the boundary was counted as a successful trial in the online control experiment.
The other one was an obstacle-avoiding track (270 mm (W) * 750 mm (L)) (Fig. 4b). Controllers performed the online control of the cyborg to finish walking along the track from one side to the other side without crossing obstacles (red dots) on the sheet. Walking through the entire obstacle-avoiding track without touching any obstacles was counted as a successful trial.
Besides the above single-control experiments, we also conducted a double-control experiment to primarily explore the possibility of applying the designed system to further entertainment in daily life (Fig. 5). In this experiment, two human subjects each took control of a cyborg to complete walking along the S-shape track (the same size as in single-control paradigm) in the form of competition. Two cyborgs started moving forward in the same time, and between the two controllers, the one who navigated the cyborg to walk inside the track and reach the finish line first won the trial. Ten trials in total were completed in this experimental paradigm.
3 Results
During the online control experiments, the cyborgs could produce quick responses to the applied invasive neural stimulation, and the time measured from sending a command from SSVEP to the completion of the cyborg’s reaction was about 772 ms in the present system. The cyborgs showed accurate response to the applied stimulation in most cases as well, and the mean response accuracy to the control commands reached 89.5 ± 15%. On the human side, average classification accuracy of SSVEP across three subjects in training sessions reached 86.0 ± 10.4%, indicating that both the BCI and neural stimulation subsystems in this study have the possibility to be used in an online control system.
The experimental results showed that all subjects could navigate all cyborgs to accomplish walking along both the different two tracks. The mean success rate for the online experiments achieved with this system was 20% for the S-shape track. Although not especially high, it was significantly higher than the value achieved in the control groups (0%) (t = 3.464, P = 0.0085). When using the obstacle-avoiding track, the success rate of online control could reach 40% (Fig. 4b). A demonstration video of a successful navigation of S-shape track is available through the link (https://www.youtube.com/watch?v=k5t6WkTkJkA).
An entertainment contest (double-control experiment) was also conducted, in which two persons competed to control two cockroaches, respectively. In this interesting paradigm, the success rate of navigation for each subject was the same as that in the single-control experiment (~20%). If both subjects were required to completely and successfully navigate in the same trial, this formidable task was still possible, but the average success rate was very low (~5%). Generally using the cyborgs that were more sensitive to the micro-stimulation would be more likely to win the contest.
4 Discussion
We presented a feasible method to navigate a cyborg with human brain in this study. The human subjects could successfully steer cockroaches to make desired turns with an SSVEP-based BCI and micro invasive neural stimulation. Using only 4 channels on the scalp of visual cortex with a portable EEG device, the SSVEP achieved high classification accuracy. However, misclassification appeared frequently in the SSVEP during the shifting period from one intention to another intention. Therefore, developing a program to detect transitions of the EEG signal to different frequencies before classification in further studies may help the SSVEP achieve more accurate classification and can be used in the asynchronous BCIs as well.
The performance of the online control varied among subjects and cyborgs. The biological factors of human and cockroaches affected the success rate of the present system, which actually cannot be easily solved by current technologies. Latif et al. [17] tried to steer a cockroach manually to walk along an S-shape line and finally achieved a 10% success rate. However, navigating a cyborg successfully with human brain signals in this study is much more challenging, which requires continuously high level of accuracy from both sides of “controller” (human) and “receiver” (cyborg), and calls for constantly quick response from both human intention recognition and cyborg reaction as well.
From another point of view, some other factors related to the online control task may influence the control performance as well, such as constraint level of the tracks used, BCI skills and experience from the human and so on. The result shows that the success rate of online navigation increases from 20 to 40% when switching the S-shape track to the obstacle-avoiding track that allows more freedom of control. It demonstrates well that the task and experimental paradigm can also affect the success rate.
All these factors account for the success rate, which is not high enough even though both the “controller” and the “receiver” have a relatively high accuracy of information translation for a single decision. The double-control experiment is very interesting, but this protocol, which requires two human subjects to control two cyborgs to hit the finish line in sequence, is a rather high standard in current BCI applications. It is more complicated than the single-control experiments, and we will go on exploring this issue in future study. To some extent, the performance of online control with current system is reasonable and instructive.
At present, we are developing a new type of wireless microstimulator with multiple stimulation channels and modes, which can technically contribute to the improvement of the control performance as well.
This work has realized the idea on utilizing the brain signals to steer a cyborg continuously for the first time. The idea may be used for detection in complex and dangerous environments in far future. Most importantly, this study also succeeds in building up an embryonic virtual brain-to-brain interface (BBI) to functionally transfer information from one brain to another [18,19,20]. With nogoing efforts from researchers, we believe that more modalities of both BCI and cyborg technologies will be developed and used in a variety of ways in the future.
References
A. Bozkurt, R. Gilmour, D. Stern, A. Lal, MEMS based bioelectronic neuromuscular interfaces for insect cyborg flight control (IEEE, 2008), pp. 160–163
H. Sato, C. Berry, Y. Peeri, E. Baghoomian, B. Casey et al., Remote radio control of insect flight. Frontiers in integrative neuroscience 3, 24 (2008)
H. Sato, C.W. Berry, B.E. Casey, G. Lavella, Y. Yao et al., A cyborg beetle: insect flight control through an implantable, tetherless microsystem (IEEE, 2008), pp. 164–167
H. Sato, Y. Peeri, E. Baghoomian, C. Berry, M. Maharbiz, Radio-controlled cyborg beetles: a radio-frequency system for insect neural flight control (IEEE, 2009), pp. 216–219
S.K. Talwar, S. Xu, E.S. Hawley, S.A. Weiss, K.A. Moxon et al., Behavioural neuroscience: rat navigation guided by remote control. Nature 417, 37–38 (2002)
E.S. Krames, P.H. Peckham, A. Rezai, F. Aboelsaad, chapter 1—What is neuromodulation? in Neuromodulation, ed. by E.S.K.H.P.R. Rezai (San Diego, Academic Press, 2009), pp. 3–8
J. Okada, Cockroach antennae. Scholarpedia 4, 6842 (2009)
C. Comer, Y. Baba, Active touch in orthopteroid insects: behaviours, multisensory substrates and evolution. Philos. Trans. Royal Soc. B: Biol. Sci. 366, 3006–3015 (2011)
C. Comer, L. Parks, M. Halvorsen, A. Breese-Terteling, The antennal system and cockroach evasive behavior. II. Stimulus identification and localization are separable antennal functions. J. Comp. Physiol. A. 189, 97–103 (2003)
S. Ye, C.M. Comer, Correspondence of escape-turning behavior with activity of descending mechanosensory interneurons in the cockroach, Periplaneta americana. J. Neurosci. 16, 5844–5853 (1996)
J.R. Wolpaw, N. Birbaumer, D.J. McFarland, G. Pfurtscheller, T.M. Vaughan, Brain–computer interfaces for communication and control. Clin. Neurophysiol. 113, 767–791 (2002)
B. Graimann, B. Allison, G. Pfurtscheller, Brain–computer interfaces: A gentle introduction (Springer, Brain-Computer Interfaces, 2010), pp. 1–27
J.J. Shih, D.J. Krusienski, J.R. Wolpaw, in Brain-Computer Interfaces in Medicine (Elsevier, 2012), pp. 268–279
B.Z. Allison, E.W. Wolpaw, J.R. Wolpaw, Brain–computer interface systems: progress and prospects. Expert Rev. Med. Devices 4, 463–474 (2007)
G. Bin, X. Gao, Z. Yan, B. Hong, S. Gao, An online multi-channel SSVEP-based brain–computer interface using a canonical correlation analysis method. J. Neural Eng. 6, 046002 (2009)
X. Gao, D. Xu, M. Cheng, S. Gao, A BCI-based environmental controller for the motion-disabled. IEEE Trans. Neural Syst. Rehabil. Eng. 11, 137–140 (2003)
T. Latif, A. Bozkurt, Line following terrestrial insect biobots (2011), pp. 972–975
M. Pais-Vieira, M. Lebedev, C. Kunicki, J. Wang, M. Nicolelis, A brain-to-brain interface for real-time sharing of sensorimotor information. Sci. Rep. 3, 1319 (2012)
R.P. Rao, A. Stocco, M. Bryan, D. Sarma, T.M. Youngquist et al., A direct brain-to-brain interface in humans. PLoS ONE 9, e111332 (2014)
S.-S. Yoo, H. Kim, E. Filandrianos, S.J. Taghados, S. Park, Non-invasive brain-to-brain interface (BBI): establishing functional links between two brains. PLoS ONE 8, e60410 (2013)
Acknowledgements
We wish to acknowledge the human and cockroach subjects from our laboratory, both past and present. In particular, we thank Jinming Zhang for assisting us in the experiments and preparing some of the figures.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2017 The Author(s)
About this chapter
Cite this chapter
Li, G., Zhang, D. (2017). Brain-Computer Interface Controlling Cyborg: A Functional Brain-to-Brain Interface Between Human and Cockroach. In: Guger, C., Allison, B., Ushiba, J. (eds) Brain-Computer Interface Research. SpringerBriefs in Electrical and Computer Engineering. Springer, Cham. https://doi.org/10.1007/978-3-319-57132-4_6
Download citation
DOI: https://doi.org/10.1007/978-3-319-57132-4_6
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-57131-7
Online ISBN: 978-3-319-57132-4
eBook Packages: Computer ScienceComputer Science (R0)