Keywords

These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

1 Introduction

Wisdom Web of Things (W2T) provides a social-cyber-physical space for all human communications and activities, in which big data are used as a bridge to connect relevant aspects of humans, computers, and things [1]. It is a trend to integrate brain big data and human behavior big data in the social-cyber-physical space for realizing the harmonious symbiosis of humans, computers and things. Brain informatics provides the key technique to implement such an attempt by offering informatics-enabled brain studies and applications in the social-cyber-physical space, which can be regarded as a brain big data cycle [2]. This brain big data cycle is implemented by various processing, interpreting, and integrating multiple forms of brain big data obtained from molecular level to neuronal circuitry level. The implementation would involve the use of advanced neuroimaging technologies, including functional Magnetic Resonance Imaging (fMRI), Magnetoencephalography (MEG), Electroencephalography (EEG), functional Near-Infrared Spectroscopy (fNIRS), Positron Emission Tomography (PET), as well as other sources like eye-tracking and wearable, portable, micro and nano devices. Such brain big data will not only help scientists improve their understanding of human thinking, learning, decision-making, emotion, memory, and social behavior, but also help cure diseases, serve mental health-care and well-being, as well as develop brain inspired intelligent technologies to provide wisdom services in the social-cyber-physical space.

2 Developing a Big Data Sharing Mind on the W2T

Currently, various Internet of Things/Web of Things (IoT/WoT), and cloud computing based applications accelerate the amalgamation among the social, cyber and physical worlds (namely a social-cyber-physical space). As shown in Fig. 15.1, the wisdom Web of Things (W2T) has been developing as an extension of the wisdom Web in the social-cyber-physical space with big data [1]. The wisdom means that each of things in the IoT/WoT can be aware of both itself and others to provide the right service for the right object at a right time and context. Furthermore, WaaS (Wisdom as a Service) has been proposed as a content architecture of the big data cycle and a perspective of W2T in services for the large-scale converging of big data applications on the W2T [3]. In other words, WaaS is an open and interoperable intelligence service architecture for contents of IT applications, i.e., data, information, knowledge, and wisdom (DIKW). The social-cyber-physical space with its big data cycle would serve this purpose. Because of the fusion of humans, computers, and things in the social-cyber-physical space, today we live within a huge network of numerous computing devices, measuring devices, and u-things, where real physical objects are attached, embedded, or blended with computers, networks, or some other devices such as sensors. Adapting and utilizing this kind of new human-machine relationship and developing human-level collective intelligence become a tangible goal of the DIKW related research. Realizing it will depend on a holistic intelligence research with two ways. On one hand, human brain needs to be investigated in depth as information processing system with big data for understanding the nature of intelligence and limits of human brain [4]; on the other hand, brain big data are collected in the social-cyber-physical space and integrated with human behavior big data and worldwide knowledge bases to realize human level collective intelligence as a big data sharing mind a harmonized collectivity of consciousness [5].

Fig. 15.1
figure 1

The framework of wisdom Web of Things (W2T) in the social-cyber-physical space

Full size image

If we could apply this concept to the Shannon-Weaver model of communication in the social-cyber-physical space, the information in the senders mind would be converted into data, and in this process the sender would take what is necessary and discard what is not. The sender’s data would be transferred through channels and received by the receiver. The receiver would decode the data and interpret the meaning of the data and generate new information in his/her mind. Thus, there is no direct relation between the information of the sender and that of the receiver. When we assume that the productivities of the society, community and persons are proportional to the amount of information transferred, we can introduce the factor of efficiency that can be expressed as a percentage of what ideally could be expected. In machine to machine communication, the efficiency should be 100 %, but in human to human communication, the efficiency varies from minus infinity to plus infinity.

The ultimate purpose of communication between humans is “a meaning communicated which is appropriately understood while people cooperate creatively with one another”. As information communication more high-speed, high-volume and ubiquitous while requiring more dependability, it has become more important to address qualitative problems in communication for human beings while making it possible to convey real, meaningful and understandable messages freely and appropriately without restriction.

To consider an ambiguous figure, in which some 3D information deteriorates into 2D, even if you and I see the same figure, I cannot tell what you bring up in your consciousness or how you see the figure [6]. Contrarily, when you may draw a caricature of a politician, the data size of the caricature can be compressed into much smaller than his/her high-density (HD) photos, but the effect elicited by the caricature is much larger than HD photos. When we observe a hidden figure that is a visual image degraded by monochromatic binarization, only meaningless patterns are seen for the first time while after some seconds a meaningful object is suddenly perceived [6]. These experiences raise the question: What is the essence of understanding? Even with a small amount of information, the efficiency may compensate the productivity. Especially in human-to-human communication, inspiration and inspiring creativity work quite effectively.

While considering such circumstances, heart-to-heart science (HHS) conducts research and development to assist people to understand the meaning of words and recognize the content of information by scientifically analyzing the higher-level brain functions related to “understanding of meaning”, “recognition” and “affect”, which are the core of communication. When an ambiguous and/or incomplete information is presented, a computer cannot understand the meaning. On the other hand, if you see such information, you can guess the meaning of the information by inspiration. The inspiration of awareness is a key for improvement of the efficiency of information transfer. For information communication technology (ICT), especially for communication between humans in the social-cyber-physical space, it is crucial to improve the efficiency. Brain informatics should provide opportunities for this improvement by understanding and applying how the brain identifies the “heart” of the information, as well as developing the brain inspired W2T technology for communicating only the true information, namely sending what we need to send, and receiving only what we need to receive.

3 Network Based Big Data in the Social-Cyber-Physical Space

In the relation between the neuroscience and big-data, there are several interactions. Brain function measurements generate big data, which could be used by the information networks. Sensor networks generate human behavior data, which is also big data. Both types of data offer the stimulus set for brain researches. Network science provides analysis of network for such network based big data in the social-cyber-physical space. The analysis methods will be applicable to brain network analysis [7, 8]. When we found the new topologies, brain research gives new concept of the network.

Due to the emergence of the popularity of using smartphones, ICT has impacted by many IC cards and passes with IC tags. Furthermore, wearable sensors attached to a person continuously send information of his/her health conditions, such as heart rates, blood pressure, blood glucose levels and so on, to hospitals or doctors. These technologies have accurately recorded daily lives and social behavior of individuals as digital data logs. These types of data are all big data and would be utilized for various analyses and studies. From the ethics point of view, establishing rules for utilizing big data while taking into account of privacy protection are required.

Since big data implicitly includes ensemble behavioral data of people in the social-cyber-physical space, the rules or structures of human behaviors can be extracted from them, which may reflect human brain functions. Behavior of complex dynamic systems has so extensively been studied in mathematics, biology, and complex system sciences, that combination of these studies to big data has provided some important insights of ensemble behavior of people. For neuroscience and cognitive science, thus, utilization of the big data as stimulus sets has now provided new ways to better understanding of human brain functions and mechanisms. There are several studies carried out, in which stimulus sets from big data were used and a kind of reverse engineering of the human brain was performed [9, 10].

For example, neuroeconomical studies on decision-making in a social context have revealed the network of brain regions responsible to social decision making. Many of these studies have been carried out with using behavioral economics games [11]. Using these games, our research group also aims to construct the computational model of human decision making which enables us to predict future behaviors [12]. We are particularly interested in decision making in social settings, individual differences in decision making and learning mechanism of decision making. Results obtained with these studies will provide new aspects for analysis on big data and especially on social media such as SNS or twitters.

fMRI measurements provide a large amount of nodes and connections. The relationship between the nodes is analyzed by network analysis for building the model in which the degree of freedom is autonomously reduced [13]. Brain is the ensemble of transfer functions. We would like to know the transfer functions, the way of representation of information in brain, and transition between unconscious and conscious. For these studies, images and auditory stimuli, such as languages with tags will be useful as stimulus sets. The various stimuli are applied to a subject and measure the brain activity by fMRI. Another observation is that human thought is always the product of multiple collaborating brain centers linked by white matter tracts. Thinking is a network function and white matter is the unsung hero of human thought [14, 15].

To analyze the relationship between the input and patterns of neural activities, we can estimate the transfer function with using machine learning techniques. This is an example of the usage of big data for neuroscience. Nishimoto and colleagues collected various types of natural movies from the Internet and used them as stimulus sets [16]. These movies are presented to subjects. Then, evoked patterns in visual cortex are analyzed with machine learning and the way to reconstruct visual experiences from the brain activity has finally been found.

How should we collect neural activity during daily lives? To combine the huge non-invasive measurement systems and ordinary human activity, we need to fetch daily-life environment into these measurement systems and bring the brain activity measurements into social and daily lives. Although fMRI and MEG are versatile and precise imaging methods which non-invasively measure and image human brain activities, they require electromagnetic shield and vibration isolation system for their high accuracy at reasonable temporal resolutions. These requirements make the systems far from portability and being wearable and restrain persons as subjects in these machines. In addition, fMRI and MEG are too large to be moved and the subjects are under highly constraint conditions. This situation is very far from the daily life. Therefore, we have to develop a simple and mobile measuring system of neural activity, which is combined with measurements and records of other physical activities as shown in Fig. 15.2. These form big data of human activities. An example is to develop portable EEG which requires no paste or gel for electrodes and transmits the signals measured by wireless telecommunications.

Fig. 15.2
figure 2

How would brain big data be collected and used in the social-cyber-physical space

Full size image

The other option is to establish the daily lives in the fMRI with using big data. To combine the huge non-invasive measurement systems and ordinary human activity, we need to fetch daily-life environment into these measurement systems and bring the brain activity measurements into social and daily lives. Since the MRI is very noisy, you need ear plug. Two fMRIs are connected with a tube equipped by microphones for the natural dialogue inside MRI. For visual stimulation, 3D images and movies are represented to the subject inside fMRI. Haruno and Frith used dictator games to classify subjects as prosocial or selfish and then measured their brain activity with fMRI while the subjects rated the desirability of different reward pairs for self and other on a scale from one to four [12]. These developments have paved the way for the combination between precise measurements in laboratories and activities in ordinary lives.

Fig. 15.3
figure 3

The W2T based architecture of a smart hospital service system for brain and mental disorders

Full size image

4 Brain Big Data Based Wisdom Services

To demonstrate brain big data in the W2T applications, Fig. 15.3 gives an outline of a smart hospital service system for brain and mental disorders. Based on the previous prototype of the portable brain and mental health monitoring system that has been developed to support the monitoring of brain and mental disorders [3, 17], the development of such a smart hospital service system needs to consider various system-level and content-level demands. It is necessary to effectively integrate multi-level brain and mental health big data and provide multilevel and content-oriented services for different types of users by an open and extendable mode. Such a system is based on the WaaS architecture with a variety of data acquisition devices, the brain data center and LarKC semantic cloud platform [3, 18, 19]. Three types of data need to be collected from patients in a hospital:

  • Physiological data acquisition, based on wearable systems, as the method of long-term data acquisition to collect physiological data of patients;

  • Behavior data acquisition, based on wearable and monitoring systems, as the method of long-term data acquisition to collect behavior data of patients;

  • Data acquisition from traditional diagnosis and physiological measuring of patients, centered on scale rating, as the method of non-periodic data acquisition to collect psychological and physiological data of patients.

Various wearable health devices, such as the wearable EEG belt, wearable voice, wearable heart rate and tremor, wearable sleeping monitoring system, are used as new physical examination devices to obtain macro and meso levels of brain and mental data. These wearable health data are analyzed and integrated with clinical physical examination data, as well as various medical information and knowledge sources, including medical records, experimental reports, LOD (Linked Open Data) medical datasets, such as ICD-10, SNOMED CT, PubMed, and DrugBank. Furthermore, these integrated sources are combined with personalized models of patients for providing DaaS (Data as a Service), IaaS (Information as a Service), KaaS (Knowledge as a Service) and WaaS (Wisdom as a Service) to various types of users [3].

A powerful brain data center needs to be developed on the W2T as the global platform to support the whole systematic brain informatics research process and real-world applications [2, 19]. As the core of brain big data cycle system, the Data-Brain represents a radically new way of storing and sharing data, information and knowledge, as well as enables high speed, distributed, large-scale, multi-granularity and multi-modal analysis and computation on the W2T [19–21]. A multi-dimension framework based on the ontological modeling approach has been developed to implement such a Data-Brain [19].

Emotional robotic individual views emotion and cognition as a starting point for the development of robotic information processing and personalized human-robot interaction on the W2T data cycle system [1, 22]. At first, a cyber-individual needs to be created by collecting brain big data and social behavior data from a specific user, in addition to target robotic emotional and cognitive capabilities, including perception processing, attention allocation, anticipation, planning, complex motor coordination, reasoning about other agents and perhaps even about their own mental states [22, 23]. Then an emotional robotic individual embodies the behavior of a user in the physical world (or the cyber world, in the case of simulated cognitive robotics). Ultimately the robot must be able to act in the real world and interactive with a specific user, such as a patient with depression, to help his/her psychological treatment and rehabilitation.

5 Five Guiding Principles

To prepare well for the massive progresses in brain big data in the social-cyber-physical space, technological innovation is necessary but not sufficient, we do need to have a deeper understanding of how technically brain/behavioral data can(not) be collected and the nature of human perception/behavior. To this end, we propose five guiding principles.

First, “dynamic link across brain-body-world” is the key. Just as an example, the “gaze cascade” effect illustrates how gaze shift interacts with perceptual processing and facilitation to form a dynamic positive feedback loop towards a conscious decision of visual preference [24].

Second, the implicit cognitive process (of “tacit knowing”) needs to be understood. The brain/mind processes that are consciously aware of is just a tip of iceberg—the vast majority of neural processing remains implicit, including the critical mechanisms underlying decision making. To prove this, we showed that the nucleus accumbens (a subcortical structure well known to be a part of the dopaminergic cirtuit) is activated to the more preferable face image than the less, not only in the explicit preference judgment task but also in an emotionally-neutral, control task on the same face images (i.e. to judge which face is rounder).

Third, perception and action are interactive & ubiquitous from the outset. Most directly demonstrating this point is the well-known snake illusion. In this and several related illusions, an entirely static image yields a strong impression of motion (rotations, in this case), typically contingent upon saccadic eye movements. While the critical factors and the underlying mechanisms are still under a debate [25], it is obvious that active approach from the observer to the object triggers a vigorous interaction to yield dynamic perception, and this is true everywhere even in the natural (non-technological) environment. As for yet another line of evidence, there are studies indicating that a locomotion or a body movement vigorously changes perception [26].

Fourth, predicting-past is easy, predicting-future is hard. This is so fundamentally because the brain and behavior/decision of the human is strongly situation-dependent. There are many lines of evidence, mostly in behavioral, EEG, and fMRI studies, indicating that the data obtained from the same brain with the same stimulus materials/tasks are necessary to perform reliable decoding, with saying larger than 90 % correct rate, including our own EEG study of decoding facial preference decision [27]. One may also expect a reasonable and feasible ethical border here, to deal with related ethical issues in the near future.

Fifth and finally, we would like to emphasize that creativity is “waiting” in the environment. Just to intuitively illustrate this point, imagine the following (gedanken) folktale. Two towns were facing against each other beyond a very deep valley. People travel miles of mountain road to trade with each other. Constructing a bridge was an obvious solution, but until one rich man came up with the idea and actually implemented it, nobody ever thought of it, so one may call it a creative insight. One may add that this was the first bridge ever created in the human history, to make it more persuasive. But at the same time, one may also argue that the environment (i.e. landscape) had been structured such that this creative idea was implicitly awaited.

To conclude, technical attempts and discussion around brain big data should be based on the keen realization of the vigorous interaction and interdependence of brain-body-environment.