Keywords

1 Introduction

Last years the very promising approaches in human-computer interaction (HCI) were investigated: speech recognition, movements, postures and gestures [1]. One of the new methods of HCI is brain-computer interface (BCI).

In recent years, BCI devices have become commercially available. Therefore, many new opportunities have become available to developers and researchers to create fundamentally new applications based on such devices.

One of such applications could be a hardware and software system that offers attention training programs for children with Attention Deficit Hyperactivity Disorder (ADHD). The prevalence of this psychological disorder makes it necessary to develop effective methods of therapy. One of the challenges researchers face is how to make a child genuinely interested in an object.

The main feature of this solution is its combination of the BCI and the humanoid robot NAO. The process of attention training is game-based.

2 Background

2.1 Attention Deficit Hyperactivity Disorder

ADHD encompasses a wide range of symptoms, but is most often associated with inattention and hyperactivity that interfere with one’s ability to do everyday tasks. ADHD often leads to decreased performance in school during childhood, and difficulties working in adulthood.

According to the statistics, approximately 4 to 18 % of Russian children have ADHD, in the US this figure is 4–20 %, 1–3 % in the UK, 3–10 % in Italy, 1–13 % in China, 7–10 % in Australia. Boys are diagnosed with ADHD, approximately nine times more often than girls. It’s important that ADHD symptoms are found not only in children, but also in adults. For example, 3–5 % of adults suffer from ADHD in the US. In 30–70 % of cases the ADHD symptoms remain for the whole life [2].

From the neurological point of view ADHD is a stable and chronic syndrome with no established treatment. However, most experts consider that the most effective approach is the combination of several methods that should be selected individually for each case. The methods of behavior modification, psychotherapy, educational and neuropsychological correction are used [3]. One of the methods of non-drug treatment for ADHD is a therapy based on the biofeedback (BFB). Biofeedback technique has been used to treat ADHD for over 30 years. Active participation of a child in treatment is one of the advantages of this method [4]. In the therapeutic phase a patient is taught the technique of brain activity self-correction using adaptive feedback where visualized EEG characteristics are the controlled parameter [5].

In 2012 Lim et al. showed that the 8-week training children through games using BCI led to a significant improvement in inattentive symptoms and hyperactive-impulsive symptoms. We decided to test whether this experiment efficiently if we add to the gameplay something bright, eye-catching. This element became a humanoid robot that looks like a child.

2.2 Global Software Engineering

From October 2013 and January 2014 Ritsumeikan University and Tomsk State University of Control Systems and Radioelectronics (TUSUR) held a collaborative Japanese-Russian university course in Global Software Engineering (GSE). Lectures were delivered by Victor Kryssanov (Ritsumeikan University), Tomasz Rutkowski (University of Tsukuba) and Evgeny Shandarov (TUSUR) via teleconference. The goal of the practical part of the course was to organize a group of students with different majors to execute the final project. Some of the students were located in Tomsk (Russia), and others were in Kyoto (Japan), so all discussions and collaboration within projects took place in social networks. During this course, students have been developing scenarios of human-robot interaction by BCI Emotiv EPOC. The robot used was Aldebaran Robotics NAO and it was located in Tomsk, meanwhile Emotiv EPOC was located in Kyoto.

As a result of its work, one of the groups has developed a scenario for treatment of ADHD in children. Our group: Titinunt Kitrungrotsakul, Dang Tuan Linh, Boonsita Roengsamut, Peeraphan Puttawetmongkol, Zhu Shuaizhen, Egor Sidorov, Gomilko Stepan, Dashkevich Mikhail.

2.3 Emotiv EPOC

Emotiv EPOC is a device that implements the BCI functions. It is a helmet with electrodes which read electroencephalography (EEG) of the user’s mental activity. Emotiv EPOC has 14 sensors plus 2 references offer optimal positioning for accurate spatial resolution, gyroscope, Wi-Fi module and USB interface. Signals were analyzed by the BCI 2000. The P300 speller included in BCI 2000 is a BCI application with which it is possible to input a letter of the alphabet using human thought. P300ClassifierGUI is used for the analysis of collecting data together with P300 Speller [6].

This technology can be used for disabled patients, such as controlling an electric wheelchair, mind-keyboard, or playing a hands-free game. BCI based attention training game can be a potential new treatment for ADHD [7]. It has lower price than analogue and it has shorter preparation time to use [8]. However, before using the BCI individual calibration has to be carried out. It is important to ensure that the calibration is as quick as possible, because people with ADHD have wave P300 amplitude mostly lower than healthy persons, or extremely elevated. The amplitude of wave P300 correlates with lack of attention [9]. It should be noted that the signal pickup and command recognition requires 300 ms [6].

2.4 Aldebaran Robotics NAO

To hold the child’s attention, we should involve him in gameplay and arouse his interest. For example, we can use a bright moving toy attracting attention. Motions should not repeat in the script.

The Aldebaran Robotics Nao was used for experiment. The Nao is a research platform used by more than 550 prestigious universities and research labs around the world. It is a small humanoid robot, measuring 58 cm in height, weighing 4.3 kg and having 25 degrees of freedom. It has a range of sensors and actuators: 2 loudspeakers, 4 microphones, 2 cameras, a gyroscope, an accelerometer, and range sensors (2 IR and 2 sonars). The robot has an embedded computational core and connects externally via Wi-Fi or Ethernet. The Nao has a generally friendly and non-threatening appearance, which is therefore particularly well suited for child-robot interaction [10].

3 Robot-Human Interaction Scenario and Experiment

Based on the review, the following robot-human interaction scenario was proposed. The BCI Emotive EPOC is used by child. The robot explains the rules of the game to the child by voice. The robot voice synthesized by ALTextToSpeech module of NAOqi framework. Then the robot demonstrates a sequence of four activities (e.g. go forward, backward, left, right) to the child. After that the robot asks the child to “repeat” the sequence of ‘thinking’ commands to the robot. Commands are formed by BCI. The robot executes these commands one by one and if the command is correct, the robot waits for the next command, and if not, it asks to try again. In the end, the robot congratulates a child and offers to play again.

The main goal of the proposed game is to train child to hold attention. Cycle of the game will completed successfully when the initial sequence of activities will fully performed by robot.

The diagram of the system is shown in Fig. 1. The BCI part of the work was carried out by students in Kyoto. The robot executing the commands was in Tomsk. Communication between components of the system was conducted over HTTP on the public Internet networks. Client and server components communicate via a developed protocol. Protocol commands were transmitted to the server using the GET method in the format http://<server address>/set_command.php?<Command>. Command set included F (NAO moves forward), B (NAO moves back), R (NAO moves right), L (NAO moves left).

Fig. 1.
figure 1

Diagram of the system

Full size image

The robot polls the server every two seconds for the command. The user interface is shown in Fig. 1 in P300 Speller section.

As we mentioned above, all work related to BCI carried out by our colleagues in Kyoto. Group in Tomsk has developed server software and robot software (Fig. 1). Choregraphe tool from Aldebaran Robotics was used to develope robot software and motions. This is a visual development environment that allows you to program robot and to create natural motions for NAO. We had used Python to develop some of parts of code. PHP was used to develop server software.

Preliminary testing of the system we carried out by hand forming commands. Collaborative experiment was in January 2014. Microsoft Skype was used to ensure visual contact research groups.

At the time of the experiment, the team has created five sequences of motions for the robot. Experiments were carried out for two hours. During this time, four complete cycles of the game have been successfully run. The main flaw of the system was a significant time interval between the beginning of the command formation and its execution by NAO, which was about 15–20 s. The main reasons for this were long delay in reading, analyzing and sending commands to the BCI (15–20 s) and delay in polling of the server by robot (2 s).

4 Conclusions

As a result of this work the team has proposed the system concept for training attention, implemented a prototype game and performed tests of the system via the public Internet networks. The main problem was the long delay between commands. Given the success of the experiments of Lim and others [7], this scenario may be used in further experiments with people suffering from ADHD. In the next work stage the therapeutic effectiveness of the prototype will be investigated.