1 Introduction

Brain–Computer Interface (BCI) is an alternative, augmentative communication channel that aims at providing the user, possibly lacking voluntary muscle control, with an interaction path based on the interpretation of his brain activity. BCI has been used to support communication [19, 48, 50], and effective, non-invasive solutions have been developed, featuring high accuracy and interaction speed. Other applications include, for example, control of mobile robots [3], orthosis [35] and artificial limbs [17], electrical wheelchair [7], functional electrical stimulation system for motor recovery [12], and many others. In general, such systems are designed for frequent and intense interaction with the user: they usually exploit expensive hardware setups and often require to involve the user in demanding training/calibration procedures.

A promising application of BCI technology is the control of Active and Assisted Living (AAL) systems [27, 30], designed to make the home environment smarter and to support the user in accomplishing daily living activity tasks. AAL systems have been shown to provide effective solutions for promoting active ageing and independent living [4, 14]. However, interacting with AAL systems could be extremely difficult, or even not possible at all, for people affected by severe motor impairments (and thus not able to control conventional interfaces); BCI could therefore provide a technological bridge to enable users with impairments to gain full access to AAL services.

Design of BCI solutions aimed at AAL control should take into account the different aspects with respect to other contexts. First of all, interactions with the AAL system (e.g. activating an appliance) are usually limited in number and sparse in time; instead, other BCI applications, such as spelling or controlling an electrical wheelchair, usually feature a frequent and almost continuous input stream. This implies frequent events and, therefore, puts tight constraints in terms of low command latency. AAL-focused solutions, instead, do not require such strict constraints in terms of speed, allowing to focus more on robustness and accuracy.

Likewise, accessibility and affordability are major concerns here: “Plug&Play” approaches, requiring neither an initial calibration phase, nor a special user training, are preferred over complex and demanding deployment procedures. Such basic differences (with respect to more conventional BCI application) justifies the customization of a BCI-service better suiting the needs. Hence, a custom BCI framework, named Brain.me, has been developed, aiming at providing a complete solution focusing on accessibility, compactness and cost reduction.

Brain.me exploits electroencephalographic (EEG) signals: among many brain imaging techniques, such as functional magnetic resonance imaging (fMRI [1, 44]) or near infrared spectroscopy (NIRS [9, 42, 43]), EEG offered us the best overall tradeoff between compactness, spatio-temporal resolution and costs [41].

The system is composed of three main units:

  1. 1.

    an analog front end (AFE) unit (Fig. 1), for the acquisition of the EEG signal;

    Fig. 1
    figure 1

    a A photo of the 16 channel analog front end (AFE) module. Dimensions: 130 × 100 mm, b a high-level module schematic

    Full size image
  2. 2.

    a digital signal processing unit, implementing feature extraction and classification;

  3. 3.

    an output/feedback unit for display and implementation of active controls.

This paper describes the Brain.me platform, focusing mostly at the development of core BCI algorithms and steady SSVEP (steady-state visual evoked potentials) processing methods. The software platform runs on standard PC architectures; nevertheless, since cost effectiveness and portability are major factors in the targeted AAL context, all software and hardware components were conceived aiming at embedded systems implementation as well. Also, great effort was put in delivering a Plug&Play service: the user should be able to control the BCI as soon as he wears the electrode cap, without performing any system calibration.

Major innovations thus come from both the hardware and the software components:

  • A custom-made EEG module has been developed, which is shown to deliver performance comparable to commercial, high-grade EEG devices, at much lower costs.

  • A signal processing methodology has been developed for increasing the robustness of the BCI in self-paced operation, through the introduction of a prediction confidence indicator. This allows to model rest and active control periods, and to discriminate between them. It is worth remarking that the definition of the confidence indicator is derived without any subject-specific training: being a stable feature across users, it can be defined once and then shared between different subjects, thus allowing some sort of Plug&Play interaction. Furthermore, we show that such indicator also allows to introduce adaptation in the signal processing flow by dynamically choosing the optimal length of the observed signal window: this improves the BCI responsiveness, as discussed later.

The approach is demonstrated in a real AAL scenario implementing the entire control chain: the brain.me platform is exploited for real-time control of the CARDEA AAL system [4, 14], involving untrained users in elementary home control tasks and achieving remarkably low false positives rate, significantly improving over the literature findings.

2 Methods

The overall system structure is illustrated in Fig. 2 and includes several modules, ranging from the EEG acquisition module to the home control section. Main modules are described in the following.

Fig. 2
figure 2

Sketch of the Brain.me platform, featuring customized EEG acquisition unit, an output unit and a signal processing block

Full size image

2.1 Analog front end

Acquiring EEG signals is an inherently challenging task: signals have amplitude as low as a few μV, often corrupted by many noise sources, including electromagnetic or others related to user’s activities [15]. This imposes tight constraints on the electrical specifications of the acquisition system: in typical BCI setups, high-end EEG equipment is used to overcome such problems. Such an approach, however, poorly suits the AAL application’s needs, due to high costs and potentially bulky equipment. Thus, a dedicated analog front end (AFE) circuit was realized [26, 28, 31], as shown in Fig. 1. It is based on a standard 4-layer PCB (Printed circuit board) technology, designed to accommodate up to 16 input channels (Fig. 1b) within a 100 × 130 mm form factor. The circuit was conceived for being battery-operated (4 × AA alkaline batteries), and power consumption is as low as 160 mW. In order to minimize noise, special layout techniques were adopted, and dedicated low-noise components were selected, including precision voltage references, low-noise and low-dropout voltage regulators, and, finally, high resolution (24 bits) analog-to-digital converters (ADC, in this work, two paralleled Texas Instruments ADS1299 [22] chips). The adoption of low-noise, low-gain ADC allows also to improve the input dynamic range of the device: the module is capable of handling up to ±188 mV differential signals. A right leg driver circuit [45] was implemented to reduce mains interference, and inputs are guarded against RF noise sources. Good noise performance is achieved, with a total 1.4 μVpp, referred-to-input (RTI) noise level (measured with electrode connectors short-circuited together). As shown in [31], such performance figures favourably compares against a commercial, state-of-the-art module (g.tec USBamp). Standard passive Ag/AgCl electrodes are used and held in place by a light head cap. Standard (DIN 42802) safety connectors make the system compatible with other EEG devices. The device communicates over USB 2.0 to the host computer at 12 Mbps, allowing to directly control the module via software. Hardware cost is in the order of a few hundred Euros, i.e. a fraction of commercial EEG devices featuring comparable specifications.

2.2 Control software and home control unit

The control software is divided into two entities:

  • An embedded part, implemented on an ARM® Cortex®-M4 platform. Main tasks of this unit are low-level hardware management (initialization and proper power-up/operational sequencing), decoding of commands, collection, and streaming of EEG data (and synch signals such as markers).

  • A MATLAB-based part, hosted on a general purpose personal computer, which is also compatible with the Octave open-source framework. Nevertheless, this part was designed with portability and computational constraints in mind and is suitable for embedded implementations as well. In addition to ingestion and storage of incoming EEG data, this unit also takes care of streaming such data to third parties’ receivers (for instance, the platform is compatible with the TOBI [33, 39] standard input interface), as well as invoking our signal processing routines.

Figure 3 shows an example of interaction between the embedded and the PC-hosted control software in case an acquisition session is performed: both parts perform some initialization steps. Then, the MATLAB-based platform issues a command to start the acquisition, which is received and interpreted by the EEG module’s command parser. After an acknowledge message, the EEG modules streams all data collected by the ADCs, along with any other external synch signal (such as markers). Meanwhile, the control unit acquires the data, saves and process them, and, optionally, streams to other receivers. Acquisition can be stopped by the host PC at any time and the EEG module returns in a waiting state by running the command parser.

Fig. 3
figure 3

Control software main tasks: (left) EEG module control firmware, (right) MATLAB-based control software on the PC side

Full size image

Real-time data processing can be carried out concurrently: an epoch-based approach is adopted, in which algorithms are executed on segments of data (e.g. 2 s chunks, possibly overlapping). Based on classification, control feedback actions are activated, as discussed below.

In order to allow for BCI interaction, home devices, such as appliances, motorized doors, windows, curtains are controlled by the CARDEA system, described in [4, 14]. CARDEA system is flexible and scalable and allows any compatible input device to achieve control over virtually any kind of peripheral device: this makes it particularly suitable for dealing with needs of people with disabilities. The Brain.me platform can be configured as a CARDEA-compatible input device, and the BCI output can be easily linked to arbitrary system functions.

2.3 Signal processing

2.3.1 Paradigm choice

Several features in the brain activity can be used to this purpose: the voluntary modulation/control by the user of such features and their consequent interpretation is called a paradigm. Most diffused paradigms include:

  • Slow Cortical Potentials (SCP), which are potential shifts in the EEG waves voluntarily induced by user, who can learn to control them through neurofeedback-like approaches [16].

  • Event Related Desynchronization (ERD) and ER Synchronization (ERS) [20, 37, 38, 47]: this paradigm exploits the brain response arising when preparing (or just imagining) to start a movement. In such conditions, neurons tend to de-synchronize from their idle state, to be allocated to motor processing: this is reflected as a decrease in spectral energy in the μ and β bands (8–12 Hz and around 20 Hz, respectively). After ERD, a pattern with an increase in the energy band after the completion of the motor task can also be observed (ERS).

  • P300 [6, 10, 18, 34, 40, 51]: when a rare target stimulus is presented to the user during a sequence of repetitive, non-target stimuli, a characteristic pattern can be observed in the EEG signals, approximately after 300 ms from the target stimulus appearance.

  • Steady-State Visual Evoked Potentials (SSVEP) [2, 8, 11, 13, 17, 21, 29, 35, 48],: this paradigm exploits features elicited by the involuntary response to a continuous, repetitive stimulus, such as a blinking LED (light-emitting diode). Within a typical 4–40 Hz range, the blinking frequency induces a synchronous tone in the brain signal spectrum. By simultaneously presenting multiple visual stimuli, each one operating at a different frequency, the user’s brain response can be analysed to infer to which stimulus he was attending. This allows to implement a multiclass (i.e. target) selection mechanism for the BCI. Furthermore, SSVEP is just a particular class of steady-state evoked potentials. Steady-state evoke potentials (SSEP) can also be elicited by means of, for example, vibrotactile stimulation [5]. SSVEP responses are largely exploited in applications where information transfer rate (ITR, defined in [25, 50]) maximization is of primary concern, e.g. in spellers.

Given the specific application aim, we hence selected the SSVEP paradigm for Brain.me operation. In fact, SSVEP responses are regarded as reliable features for BCIs, due to their higher SNR (Signal to Noise Ratio) [8]. Moreover, SSVEP detection and classification can be carried out without any calibration data [13, 21, 29, 32, 48], paving the way for a “Plug&Play” user interaction.

2.3.2 Classification

According to the selected SSVEP paradigm, user’s intention is expressed by selecting one between many different visual flickering stimuli (each one with different frequency). Classifying a SSVEP, thus, basically implies recognizing the hidden stimulus frequency.

Many algorithms exist for SSVEP classification, including PSDA (power spectral density analysis), MEC (minimum energy combination) [48], AMCC (average maximum contrast combination) [13], CCA (canonical correlation analysis) [21].

Power spectral density analysis methods rely on the estimation of the spectral components of the EEG spectrum, usually via Welch’s modified periodgram [26]. In this case, the power spectrum of each EEG channel is estimated by means of a periodgram over a short interval (the signal is windowed, usually exploiting Hamming windows):

$$P_{xx} (k) = \frac{1}{{NF_{\text{s}} }}\left| {\;\sum\limits_{n = 0}^{N - 1} {x_{n} {\text{e}}^{ - j2\pi kn/N} } \;} \right|^{2} \;,$$
(1)

where N is the window length, F s the sampling frequency, x n the nth sample. Then, averages are evaluated over a moving window (potentially with overlapping epochs) in order to lower the variance of the estimate. A search for peaks in the EEG spectrum can be performed subsequently, associated to SSVEP responses. Some heuristics can be added as well, in order to improve the accuracy of the classification. An implementation example can be found in [26].

Another possibility is to directly model the SSVEP response as follows:

$$y_{i} \left( t \right) = \sum\limits_{k = 1}^{{N_{\text{h}} }} {\left( {a_{k} \sin 2\pi kft + b_{k} \cos 2\pi kft} \right) + E_{i} (t)} ,$$
(2)

where y i (t) is the potential measured from a single electrode, E i (t) is a noise and nuisance signal, and the first term in the right-hand side is the model of a SSVEP response (a k and b k are coefficients from signal decomposition); this term corresponds to a stimulus frequency f (k = 1, …, N h, where N h is the number of considered harmonics). Extending to multiple (N t ) channels, Eq. (2) can be re-written in matrix form: Y = X + E, where Y are the signals in the electrode space, E represents channel noise, X is the SSVEP model matrix (there is one such matrix per stimulus frequency):

$$X = \left[ {\begin{array}{*{20}c} {\sin 2\pi ft_{1} } & {\cos 2\pi ft_{1} } & \cdots & {\sin 2\pi N_{\text{h}} ft_{1} } & {\cos 2\pi N_{\text{h}} ft_{1} } \\ \vdots & \vdots & \vdots & \vdots & \vdots \\ {\sin 2\pi ft_{{N{}_{t}^{{}} }} } & {\cos 2\pi ft_{{N_{t} }} } & \cdots & {\sin 2\pi N_{\text{h}} ft_{{N_{t} }} } & {\cos 2\pi N_{\text{h}} ft_{{N_{t} }} } \\ \end{array} } \right].$$
(3)

To this purpose, we introduce a spatial filter, i.e. a filter that transforms the input waveforms from the electrode space Y (the original electrode readings) to a signal space S:

$$S = Y\;W,$$
(4)

where W is a weight matrix, which is determined by optimizing with respect to a given criterion. The number of channels in the signal space does not necessarily matches the number of channels in the electrode space.

For example, MEC determines such a weight matrix by minimizing the energy associated to noise \(\tilde{\varvec{Y}}\), estimated as follows:

$$\tilde{Y} = Y - X\left( {X^{T} X} \right)^{ - 1} X^{T} Y\;.$$
(5)

In (5), the orthogonal projection is used to remove any potential SSVEP activity from the original signals. As anticipated, one then finds an optimal set of weights ŵ that minimizes the energy of the nuisance signals \(\tilde{\varvec{Y}}\):

$$\hat{w} = \mathop {\arg \hbox{min} }\limits_{{\hat{w}}} \,\,\left\| {\tilde{Y}\hat{w}} \right\|^{2} = \mathop {\arg \hbox{min} }\limits_{{\hat{w}}} \;\hat{w}^{T} \tilde{Y}^{T} \tilde{Y}\hat{w}$$
(6)

The set of optimal weights are found to be the eigenvectors corresponding to the smallest eigenvalues of the of the matrix \(\tilde{\varvec{Y}}^{T} \tilde{\varvec{Y}}\).

AMCC algorithm follows a similar path but finds an optimal spatial filter ŵ which maximizes the following generalized Rayleigh quotient between the signal and noise covariance matrices C Y and \(\varvec{C}_{{\tilde{\varvec{Y}}}}\) as in (7):

$$\hat{w} = \mathop {\arg \hbox{max} }\limits_{{\hat{w}}} \;\frac{{\hat{w}^{T} C_{Y} \hat{w}}}{{\hat{w}^{T} C_{{\tilde{Y}}} \hat{w}}}$$
(7)

CCA is another method that has been largely used in SSVEP detection and classification, since it provides a bounded output value (i.e. the correlation coefficient), which makes it easy to perform relative comparisons. The present work builds on top of this method and develops a methodology for assessing its reliability for SSVEP classification.

CCA attempts to find optimal spatial filters that maximize the correlation between two sets of variables. In particular, they look for a pair of linear combinations (canonical variables, characterized by weight vectors w x , w y ) for the two sets, such that the correlation between the two linear combinations x L  = w T x X and y L  = w T y Y is maximized:

$$\mathop {\hbox{max} }\limits_{{w_{x} ,w_{y} }} \;\rho = \frac{{E\left[ {x_{L} y_{L}^{T} } \right]}}{{\sqrt {E[x_{L} x_{L}^{T} ]E[y_{L} y_{L}^{T} ]} }} = \frac{{w_{x}^{T} XY^{T} w_{y} }}{{\sqrt {w_{x}^{T} XX^{T} w_{x} w_{y}^{T} YY^{T} w_{y} } }},$$
(8)

where X and Y are the input signal and the SSVEP template, respectively. Then a second pair is found, uncorrelated with the first one and having the second highest correlation. The process continues until the number of pairs of canonical variables equals the number of variables in the smallest set.

In the most standard approach [21], CCA is used to maximize the correlation between the input electrode waveforms and each of the SSVEP model matrices X. A basic classifier could then simply pick the largest correlation coefficient among the ones generated by the X matrices.

This work lays its bases on the CCA methodology and builds improvements on top of it to carry out online, self-paced SSVEP classification. In fact, in [32], it was shown that CCA-based methods could outperform both MEC and AMCC in terms of computational efficiency. This is particularly appealing, envisioning a future embedded implementation of the whole platform.

2.4 Confidence indicator

CCA (and all the methods presented in Sect. 2.4) is based on relative evaluations, i.e. it selects the most likely class by comparison within a set of known candidate frequencies. On the one hand, this means that there is no need of calibration procedures, which makes CCA suitable for the “Plug&Play” aim. On the other hand, however, the basic classifier is still not suitable, as is, for online operation: there is no way to discriminate between epochs containing a SSVEP response (i.e. where the user wants to issue a command) and non-informative ones. This increases the risk of false positives and is potentially more critical when only sparse actual interactions are expected, as in the target AAL control application.

In this section, the notion of a prediction confidence indicator is introduced, which allows to improve the accuracy of the classification and, at the same time, makes it possible to build a model for discriminating between rest and active control phases.

In the literature, two approaches are followed to cope with the presence of idle periods:

  • a threshold can be set on the maximum correlation coefficient in order to be validated as output class;

  • the last n class outputs are averaged and a class picked if the class average passes a given threshold.

However, if, on the one hand, setting a global threshold would allow to reject epochs with low correlation with the SSVEP model, on the other hand this would require setting user-specific thresholds. For example, in [17] thresholds are determined from the accuracy performance of CCA on the test set, each stimulation frequency being trained separately. This requires a personalization for each user, which is not consistent with our approach.

Averaging (or “smoothing”) the classifier’s output is effective in reducing false positives, but much depends on the false positive rate the classifier produces: the more outputs are averaged, the lower the false positive rate is, but, on the other hand, the command latency is increased.

The novelty of our approach lies in finding a good parameter, common and stable across users, capable of indicating the confidence level of the classification [23, 24]. Whenever such parameter reflects poor confidence in the epoch classification, no classification is attempted at all; we call this no reliable decision state a neutral state. Furthermore, we refer to such features, useful to express classification reliability, as confidence indicators or d parameters. For convenience, we constrain such features in the [0–1] range, with d = 1 representing the highest confidence. It is worth remarking that parameter d is not a probability (could become one, for example, by performing a logistic regression over it, but that would require training): instead, it relates to the actual discriminability of a SSVEP (i.e. control vs. rest and discriminability between two target frequencies).

For the adopted CCA methodology, we define the d parameter as follows:

$$d = \mathop {\hbox{max} }\limits_{{f \in F_{\text{stim}} }} \;\left| {\rho_{\text{f}} } \right| - \mathop {\hbox{max} }\limits_{{f \in F_{\text{stim}} \backslash \left\{ {f_{ \hbox{max} } } \right\}}} \;\left| {\rho_{\text{f}} } \right|,$$
(9)

where ρ f is the correlation coefficient yielded by CCA as described in Eq. (8), F stim is the set of possible stimuli frequencies and f max the frequency associated to the largest ρ f. The next subsection shows that Eq. (8) defines a good confidence indicator, which can be used to make relative comparisons between alternative hypothesis (target frequencies).

2.4.1 Validation of the confidence indicator: offline dataset

First, to assess performance based on the exploitation of the confidence indicator introduced above, an offline test was performed. Four healthy volunteers (age 22–27), with normal or corrected to normal vision, with no previous BCI experience were involved (upon their informed consent signature). During the test, subjects were seated in a comfortable armchair, at approximately 1 m from the flickering visual stimulus. They were repeatedly asked to stare for 6 s at one out of 4 possible visual flickering stimuli (at 16, 18, 20, 22 Hz). The SSVEP responses were recorded from 6 scalp locations (namely position O1, O2, PO, P3, P4, CPz, reference electrode in Cz, according to the extended 10–20 electrode mounting system), using standard 10 mm Ag/AgCl disk electrodes with conductive gel. EEG was acquired by the module introduced above, sampling inputs at 250 samples per second (SPS).

In order to show the advantage of using the d parameter (Eq. 9) for discriminating between alternative hypotheses (targets), let us consider a basic CCA classifier. Given an EEG epoch containing a SSVEP, it selects the frequency yielding the largest ρ, computed as per Eq. (8). Also, given the output choice, we compute the related d parameter as in Eq. (9). Then, we summarize such information by means of histograms, as shown in Fig. 4. Here, the outputs are partitioned by hits and errors (of the basic CCA classifier) and binned into uniform d parameter intervals. Distributions are normalized to the sample size. An ideal behaviour should associate all errors to lower values of d, with correct classification lying instead in the higher d range. Having hits and errors grouped at opposite ends of the d range would make it easy to discriminate between reliable and unreliable classifications by means of an optimally chosen threshold d*. In that case, if d > d*, a decision should be made by picking the frequency yielding the maximum value of Eq. (9). If d < d*, instead, the likelihood of making a mistake is larger and no classification should be attempted. Summarizing, the following classification rule can be derived:

$$f_{\text{class}} = \left\{ {\begin{array}{*{20}l} {\mathop {\arg \hbox{max} }\limits_{{f \in F_{\text{stim}} }} \;\left| {\rho_{\text{f}} } \right|} \hfill & \quad {{\text{if}}\;d > d^{*} } \hfill \\ {\text{null,}} \hfill & \quad {\text{otherwise}} \hfill \\ \end{array} } \right.$$
(10)

where ρ f is the correlation coefficient as per Eq. (8), d is the d parameter defined in Eq. (9) and F stim is the set of possible stimuli frequencies. A null output refers to the condition where a classification cannot be performed reliably and is therefore inhibited.

Fig. 4
figure 4

Distribution of classifiers’ hits and errors (light grey and black, respectively; each is normalized to the sample size) as a function of the parameter d. The distributions are also plotted for different EEG window lengths

Full size image

In real data, though, such an ideal distribution is just approximated (as shown in Fig. 4): some overlap can be observed between hits and errors distribution with respect to d. Thus, different choices of the threshold value d* in Eq. (10), result in different performance tradeoffs involving accuracy, data exploitation and system responsiveness. In fact, by increasing d*, the likelihood of wrong classifications decreases. At same time, however, meaningful information may be discarded (i.e. more epochs may get rejected, despite containing a SSVEP).

In order to quantitatively assess the impact of this choice on the considered offline dataset, we define the “neutralization rate” as the fraction of rejected epochs for a given value of d*. The definition of neutralization rate is consistent for this dataset, since it only contains isolated EEG segments of SSVEP responses.

Figures 5 and 6 report ITR and accuracy, respectively, depending on the neutralization rate. In particular, ITR can be calculated as follows [8]:

$${\text{ITR}} = \left( {1 - p_{n} } \right) \cdot \left[ {\log_{2} \left( N \right) + \left( {1 - p_{\text{e}} } \right)\log_{2} \left( {1 - p_{\text{e}} } \right)} \right.\left. { + p_{\text{e}} \log_{2} \left( {\frac{{p_{\text{e}} }}{N - 1}} \right)} \right] \cdot \frac{60}{{T_{w} }},$$
(11)

where N is the number of classes (4 in this offline case), p n is the probability of the neutralization state, p e is the probability of error and T w is the EEG segment length. ITR is, thus, a global performance indicator, which combines accuracy and selection speed. Of course, ITR may decrease with increasing d* thresholds, due to the loss of information associated with more rejections of “uncertain” epochs; however, in the target AAL application scenario, accuracy is more relevant than speed itself. In particular, in order to get a more reliable controller, slower systems may be tolerated, trading speed for accuracy.

figure a
Fig. 5
figure 5

Information transfer rate (ITR) (bit/min) as a function of d parameter for different windows length

Full size image
Fig. 6
figure 6

Accuracy as a function of neutralization rate, at different values of threshold d*. The solid grey line represents the original, “raw” accuracy level, without neutralization (i.e. d* = 0). The graphs are plotted for different EEG window lengths

Full size image

Figure 6 shows how accuracy distributes, with respect to neutralization rate, for different choices of the parameter d* in Eq. (10). Each point in the scatter plots refers to a different value of the threshold d* and thus represents a different tradeoff between accuracy and data yield. As expected, by increasing d* the neutralization mechanism becomes more selective and a larger fraction of epochs is discarded. Eventually, a critical d* value is reached, beyond which all epochs are rejected and the above considerations (in the absence of valid epochs) are no longer meaningful. Thus, the plots do not necessarily cover the complete d* range: this, however, has no practical consequences, since the analysis is focused on finding practical d* values, compatible with system responsiveness and data exploitation constraints. In addition, the accuracy scatter plots can exhibit a non-monotonic trend, since the relative importance of errors located in the high-d tail increases when fewer samples (i.e. the non-rejected) are available. As a guideline, given a target accuracy, the best threshold choice is the one that guarantees a lower percentage of rejected epochs. For comparison purposes, Fig. 6 also reports (solid grey line) the baseline accuracy that the CCA algorithm can achieve without any neutralization (without checking the d parameter condition in Eq. 10). As shown, neutralization consistently improves accuracy with respect to the reference case.

Finally, in order to suit the aimed “Plug&Play” vision, it is desirable that the threshold d* does not vary between subjects: we therefore characterized the method on the test population as a whole, instead of relying on a per-user basis analysis.

We eventually found that a practical range for d* exists (typically around 0.1), which allows for a satisfactory tradeoff between error rejection and data exploitation.

2.4.2 Dynamic window length choice and online implementation

The methodology introduced in previous section can also be exploited to adapt the signal processing flow, specifically by dynamically choosing the most appropriate observation window length.

In fact, in most cases, the window length is fixed and determined a priori. Processing is then carried out using a sliding window approach, characterized by a fixed window length and a fixed increment. The window length choice, in general, impacts both the classification performance and the classification latency. Short windows result in reduced latency, but increase chances of epoch rejection; on the other hand, longer, conservative windows are more likely to yield successful classifications, at the expense of increased latency. On the other hand, the time increment between consecutive windows determines the smoothness (i.e. continuous feedback perception) of the classifier. For instance, if the time increment between subsequent windows is set at 200 ms, an update rate of 5 Hz is achieved, which is more than sufficient to be perceived as responsive.

Having defined a prediction confidence indicator in Eq. (9), we can exploit it to dynamically determine the best observation window length. This novel approach, different from most common SSVEP processing, allows to break the a priori fixed trade-off between classification latency and accuracy; instead, the trade-off can dynamically vary, depending on the window length choice policy. For real-time, self-paced operation, our strategy for determining the most appropriate EEG window length works as follows. At first, a given minimum-length EEG window is observed (e.g. 2 s); after a CCA step, if the prediction is considered reliable [according to (10)], the classification is performed. Otherwise, instead of outputting a null result, the window size is increased (we consider 1 s steps) and the same decision repeated. Eventually, the system should either output a valid classification or a null result when the maximum allowed window length is reached (5 s in our case). The procedure is reported as pseudocode in Algorithm 1.

It is even possible to accommodate for different thresholds, depending on the current window length. For example, the d* threshold could decrease for increasing window length (shorter windows are more prone to error and could benefit from a more selective threshold). In general, introducing this dynamic window choice mechanism may improve user comfort, by reducing time of attention to the visual stimuli. Finally, a smoothing stage that averages the last n classifications is added to further guard against false positive performance, at the cost of an increased latency. In the given example, averaging on 5 classification yields and extra latency of 1 s.

3 Results

Based on the ideas described above, a self-paced, online BCI, exploiting a SSVEP paradigm was implemented and tested. In order to demonstrate the practicality of the approach in an AAL context, the full control chain was implemented: home actuators were actually controllable by the BCI user to complete tasks such as switching a light or controlling a motorized shutter.

The same electrode configuration and lab environment used in the offline test described above was adopted. In this case, a total of 10 healthy volunteers (age 24–61 years, 3 females) participated in this study. None of them took part in the previous offline study, nor had any prior BCI-control experience. This is consistent with our view of a Plug&Play approach, since no calibrating session is performed: the parameters, determined in the offline study, were kept at the same value.

The AAL application scenario implies sporadic interactions, so that two main BCI goals are defined: first, discerning intentional control periods from longer “rest” phases and, then, correctly classifying intentional user choices. In order to assess the system immunity to false positive events, long idle periods were introduced, during which the subject does not make any intentional choice and is allowed to talk, shift its gaze elsewhere and, partially, move. Each user performed two runs, attempting 32 task completions in each run, which were equally distributed between 4 available choices (namely, 16, 18, 20, 22 Hz, corresponding to the following AAL commands: light on, light off, shutter closed, shutter open).

Table 1 reports the online experiment results (with mean and standard deviation), in terms of true positives, false negatives and false positive rate (TP, FN, FPR, respectively). The problem nominally involves 5 different classes: 4 target frequencies plus the rest condition. A true positive in Table 1 is defined as the event in which the user focuses on a target and the BCI correctly detects and classifies it within 8 s. A false negative, similarly, is the event in which the user attempts to achieve control by focusing on a target and his/her attempt is not detected within 8 s (the user was asked to mark this event after he does not manage to issue a command). Finally, a false positive occurs when the BCI either detects a control attempt but predicts the wrong target or when the user was resting and a command is triggered (the user was asked to mark this event as well). False positives are represented in units of min−1 in order to better stress the low occurrence of errors per minute (conversely, a percentage may result confusing, since one should fix a priori the amount of rest and control periods). Finally, for similar reasons, only FPR, TP and FN are considered as key performance indicators: in our opinion, the presence of heterogeneous classes (target stimuli and a rest class) makes it difficult, or at least not very expressive, to define other quantities. In fact, TPs and FPs are expressed as user control events (which happen at a self-paced rate), whereas FPR attempts to capture errors in the BCI operation (target misclassifications or spurious activations).

Table 1 Performance of the online, self-paced experiment (4 classes + rest/no command class)
Full size table

Good performance is achieved, especially in terms of FPR: on average, it is as low as, approximately, 0.16 min−1, with a remarkable improvement over reference data reported in [36, 39]. It is worth remarking again that all results were obtained without any subject-specific calibration, keeping all hyper-parameters (d* threshold = 0.15, smoother averaging steps = 5) constant across all users. This is consistent with the aimed Plug&Play vision.

4 Discussion

This work is focused on developing a robust methodology for SSVEP processing. Practically dealing with the targeted application (namely, BCI-enabled AAL system control) requires to address two issues: infrequent, asynchronous user interactions and the need for calibration-free operation. The former implies that the operating mode should be self-paced: the BCI must autonomously discern between periods during which the user is trying to achieve control (and, in that case, decode the intent) from others during which he/she is performing other tasks, including rest. The latter issue implies to find features and methodologies which do not depend significantly on the specific user. Another aspect worth considering, related to the choice of SSVEP as operating paradigm, comes from the need of exploiting some residual eye-moving ability; studies on independent, covert attention-based BCI are, however, being carried out [49]. Another possible drawback of SSVEP as BCI paradigm is that it is not an endogenous potential (i.e. similarly to P300, the user needs to be stimulated to elicit a specific response). In the long run, receiving flickering visual stimuli could result in user’s fatigue. This can be possibly mitigated by combining SSVEP operation with other paradigms (e.g. motor imagery, SCP) or even by looking at other bio-potentials (e.g. EMG), exploiting such “secondary” paradigms for switching on and off the BCI, and thus minimizing unnecessary visual stimuli.

Our work stems from CCA-based SSVEP classification algorithms and improves their accuracy and immunity to false positives by introducing the notion of a prediction confidence indicator (d parameter). This allows to model unreliable classification attempts and lays the bases for discerning user’s intentional control periods from rest ones (thereby addressing one of the previously mentioned issues). The methodology was firstly validated offline (Fig. 6), highlighting the benefits of rejecting uncertain epochs (according to a comparison with a target d* value). Moreover, the d parameter can be used to dynamically scale the length of the observed EEG window, making the system more responsive when enough evidence is present in the signals (and minimizing the necessary stimulus staring time, thereby improving user’s comfort). The whole system was then validated online in a self-paced scenario, keeping the same parametrization across all subject (in accordance with our subject-independent approach). Tests were performed to assess, in particular, immunity to false positives (i.e. target or rest-period misclassification): results highlighted very good performance, improving over reference works in the literature (e.g. [13, 21]).

A further proof-of-concept test was carried out in more challenging environmental conditions. The whole system (including controlled appliances) was demonstrated in the framework of an exhibition (Handimatica [29, 46]), during which 6 live demos (lasting approximately 30 min each) were carried out. A single, healthy subject (not involved in previous tests) controlled the system for more than 3 h in a harsh environment, featuring high ambient luminosity, multiple electro-magnetic interfering sources (including air conditioning high-power electric motors), with people from the audience walking around and loud acoustic noise. During the duration of demo sessions, the subject was also free to speak and move, this possibly introducing artifacts. Despite such a challenging condition, the subject was able to successfully operate the BCI, with a measured average FPR as low as 0.14 min−1 (i.e. yielding a false positive classification every 7.14 min of use, on average). Such a spot check, of course, does not exhaustively characterize and qualify the proposed approach; nevertheless, the results are quite encouraging and point in the right direction for future deployment of the Brain.me system in real living environments (such as private or sheltered homes).

5 Conclusions

In this paper, the development of the Brain.me platform is discussed. Brain.me is a low-cost solution for BCI interaction, primarily aimed at (but not restricted to) AAL system control. Such an application perspective poses quite peculiar constraints, which may differ from general purpose BCI application. A complete control chain from EEG acquisition to AAL task activation was thus implemented and demonstrated: hardware and software modules were specifically developed, focusing at low cost, portability to embedded systems and calibration-less operation. A custom EEG acquisition module has been briefly introduced, which favourably compares with reference, widely used devices.

A neutralization technique was introduced for rejecting epochs associated to low classification confidence. Offline tests show how accuracy (which is mostly relevant to the specific target application) can be significantly improved by the proposed approach. Based on the same confidence estimation, adaptive adjustment of EEG epochs duration has been suggested, aiming at improving the overall system responsiveness. This technique also allowed us to effectively implement an online BCI system capable of discerning user’s intentional control periods from rest ones.

The Brain.me platform has been exploited to implement a complete AAL control chain, including home devices actuators, and tested while working in online, self-paced operation mode. Lab experiments yielded very good results: with reference to the literature data, a meaningful improvement in FPR performance was demonstrated. Deployment of Brain.me system in harsher environments also showed good robustness and promising performance, thus moving towards effective BCI use as an alternative home control input channel.