1 Introduction

Internet of Things (IoT) consists mainly of three layers namely, (1) sensors and actuators, (2) networking and (3) applications (Mahmoud et al. 2015). IoT sensors not only include physical sensors it also consists of virtual sensors where information can be gathered from social networks which are termed as cyber physical world (Poovendran 2010; Rajkumar et al. 2010). Due to tremendous sensing capabilities and the information generated the possibility of effective use of this information is various. This leads to the complexity and concerns for participants in the IoT applications which need to be addressed. Here, we propose a solution to this problem by utilizing the usage analytics, heterogeneous sensing, data retrieval and applied machine learning that opens up a new arena of opportunity which we term it as Internet of Assistants (IoA). IoA also facilitates a human centred approach which focuses on comfort, vitality and high quality of life. To realize the IoA concept we propose two agents namely, Personal Assistant (PA) and Service Assistant (SA). The basic constructs of both the assistants are the same nevertheless the difference arises depending on service provided or consumed in the IoA environment.

PAs that makes use of machine learning and artificial intelligence are being developed to address privacy issues and complexity. These PAs are capable of making autonomous decisions on behalf of the participants with respect to smart environments using personal sensors. One such scenario is that the permission for letting the environment use the personal profile of the participant from their smartphone could be managed by PAs based on user preferences (Santos et al. 2016; Chayapathy et al. 2017). However, the contributions so far in this domain are only based on PAs to smart environment. Our proposed (IoA) consists of interconnected PAs and SAs with capabilities to collaborate autonomously within the context of preassigned privacy settings. IoA enables PAs and SAs to communicate with each other through ontologies and they can negotiate based on their preferences and privacy settings to come up with a suitable use case model.

2 Motivation and Purposes

“Humans have always been interested in understanding themselves and their environment. Understanding their relationship with the environment is important to survival as well as thriving in the present situation and planning for the future” (Jain and Jalali 2014).

The above statement kindled a fire of a plan for our current research: To design a mechanism that can match the needs of the right information required by humans and resources information collected from the environment to understand a human-environment relationship towards having suitable actions. This mechanism can make a significant change in information retrieval: humans do not need to look for information, but the information comes to humans depending on their needs.

The scope of applying this mechanism is very large and various. Applications built based on this mechanism could vary from personal to societal areas such as personal healthcare, personal assistant, situation management, smart-assistant, smart-environment, and the internet of assistants (IoA). The internet of assistants is the new terminology invented by the candidate and will be discussed later.

The proposed mechanism concerns two types of data: (1) personal data: the emergence of wearable sensors and cloud-computing has revolutionized the ability to objectively measuring physical, physiological, and mental activities of humans towards feeding data to help humans particularly understanding themselves, and (2) environment data: the non-stop development of social networks and internet of things (IoT) has provided the big data that recorded the environment status and how humans interact and perceive their surrounding environment objectively and subjectively.

Although there are many works focusing on getting insights from these types of data, few of them try to link them together to meet the needs of humans and reasonably explain causal relations among these insights. For example, human behaviour can be understood by using smartphones but there is no cue to answer whether the surrounding environment influences such behaviour or vice versa (Dao et al. 2017).

Another issue is how to choose the right data sources among heterogeneous data sources to link these types of data or to extract reasonable insights by integrating these types of data? In other words, given N sensors and M models needed to be built from N sensors, the problem is how to build M models from N sensors automatically and expert independently? For example, given N physical activities sensors (e.g. accelerometer, gyroscope), M physiological sensors (e.g. ECG, heart pulse, blood pressure), P environment sensors (e.g. rain, humidity, wind), and the context of asthma patient, how to select an optimal subset of sensors to find the correlation/association between asthma attacks, environment changing, and patient’s lifestyle? (Dao et al. 2015). Another example is presented in [4] that whether we confirm the causal relations among students’ lifestyles, emotions, and in-classroom interaction?

3 Architecture

The strategy of building this proposed system is to start from the generic framework, contextualize this framework to specific categories, get the common factors from these categories’ frameworks, refine the generic framework, and continue looping the cycle until getting the concrete and precise framework.

Then, three main categories are built based on the proposed mechanism: (1) Personal assistant, (2) Service assistant, and (3) Internet of Assistants.

3.1 Personal Assistant (PA)

Differ from existing personal assistants, the PA built by the candidate focuses on using data from physical, physiological, and mental sensors to create several models that reflect personal situations according to a spatio-temporal-context constraint. These models will help to store personal data on a semantic level instead of low-level signal data. For example, in (Dao et al. 2015) PHASOR is an instance of PA that can help humans understand their own activities objectively and automatically by feeding a stream of daily activities to their lifelog instead of accelerometer and gyroscope signal data. A PA is abstractly designed as the pair of <personal situations, needs information>_[spatio, temporal, context] where the first component represents a set of models, the second component expresses the needs of extra information from other assistants, and the index denotes the domain where the PA is established.

3.2 Service Assistant (SA)

The SA uses data gathered from the environment including physical, social, and human sensors to generate the environment situation according to a given condition. The terminology of the human sensor is used to express the data generated by humans subjectively but not from their own situation such as multimedia recorded by multimedia devices and published on cyber. The SA is abstractly designed as a pair of <environment situations, required information>_[spatio, temporal, context], where the first component represents a set of models used to understand the environment situation (e.g. weather situations = (cold, hot, wind, snow); natural disaster situations = (flood, landslide, earthquake); hospital situations = (available experts of a certain disease, vacant patient beds)), the second component expresses the required information from other assistants, and the index denotes the domain where the SA is established. For example, in (Jalali et al. 2015), the attempt of creating an instance of the SA is presented. In this work, the environment situations component contains semantic information of weather such as high temperature, strong wind, low humidity, and these states are connected by the extended finite-state automata. The required information component is expressed as a “correlation between asthma attacks and weather situation”.

3.3 Internet of Assistants (IoA)

The idea of the Internet of Assistants is to create an internet of assistants including both personal and environment assistants to help humans understand the human-environment relationships towards to survive as well as thrive in the present situation and plan for the future. In order to do this, any pair <personal assistant, service assistant> can be linked together by using the “needs information” component to activate the “required information” component to create a new node of relationships. This node itself could, in turn, be another assistant and contribute to the evolution of IoA. With IoA, independent systems developed by different sources can connect together to exchange, share, and enrich information towards enlarging the IoA. Therefore, humans can get the information they need anywhere and anytime.

4 Use Cases

4.1 A New Way for Shopping – Fast, Secure, Convenience

Mary Passes by a Shop Personal Assistant Recognize a fashion shop Checks “connection” option set by Mary If yes , start connecting to a service assistant of the shop for sharing Mary’s great taste in fashion Shop Service Assistant Capture her image Gather her great taste in fashion Send to the personal assistant list of products with prices (GAN images generated by using Mary’s image and fashion catalogues) Personal Assistant Connect to Mary’s bank Display list of products (emphasize of items whose prices she can affordable) Mary Decide to buy one item Personal Assistant Connect to Mary’s bank service assistant and the shop’s bank service assistant: transfer money Connect to Mary’s house service bank: gather time to del- very -> send to the shop’s service assistant Shop Service Assistant Connect to Delivery company’s service assistant Deliver a package to Mary’s house

See Fig. 1.

Fig. 1.
figure 1

Shopping with IoA

Full size image

4.2 A New Way for Smart Eating - Fast, Healthy and Comfortable

John Passes by a Restaurant Personal Assistant (PA) Recognize a restaurant Checks “connection” option set by John If yes , start connecting to a service assistant of the restaurant for sharing John’s eating habit Restaurant Service Assistant (R-SA) Gather his eating habit Send to the PA list of selected meals with names and cal- ories Personal Assistant Connect to John’s health service assistant ( H-SA ) Display list of meals he can eat to avoid any damage from food (e.g. allergy, diabetes) John Decide to order food and go inside Personal Assistant Connect to the R-SA to order a meal Restaurant Service Assistant Prepare the meal and a seat Personal Assistant Connect to John’s gym service assistant (G-SA) (e.g. workout’s methods) and H-SA (e.g. personal health statis- tics) to make the plan for workout John Enjoy his meal and his workout time.

See Fig. 2.

Fig. 2.
figure 2

Smart eating with IoA

Full size image

4.3 A New Way for a Smart Loan - Fast, Paperless and Convenient

John Wants to apply for a loan Personal Assistant (PA) Activate related personal information portal Bank’s Service Assistant (SA) Connects to John’s PA collect the details of John’s business connects to government’s SA for collecting further legal details about John’s business connects to the revenue department’s SA for collecting the revenue details of John’s business for previous years connect to location services’ SA for tracking the move- ent of John related to his business operations analyses all the data to provide a verdict for loan ap- proval which can be immediately processed by the manager Bank loan to John is approved

See Fig. 3.

Fig. 3.
figure 3

Loan processing using IoA

Full size image

5 Evolutional Smart Assistant Platform (ESAP)

Figure 4 illustrates the architecture of the evolutional smart assistant platform (ESAP). Green blocks denote methods, technologies, and approaches that are utilized to proceed data while blue blocks describe the output of these green blocks. The data analysis is divided into the following stage,

Fig. 4.
figure 4

Evolutional Smart Assistant Platform (ESAP) (Color figure online)

Full size image

  • Integration Discovering Analysis: Gather and aggregate data collected from sensors: In this stage, data collected from sensors or other applications and sources are integrated into a data warehouse. A Big Data system is built to handle such big data that comes from different sources. The integration of discovering analysis should be performed here to aggregate data towards decreasing the complicated diversity of data format.

  • Discover event patterns: In this stage, event patterns are discovered and extracted by utilizing up-to-date techniques of Machine learning, data mining, data system, and artificial intelligence. Here, the characteristic of the domain where such event patterns are extracted plays an important role to make sure these event patterns are useful for that domain. Discovered vent patterns are stored in the Event Ledger. An event indexing and querying schema are built here to manage events that come from different domains.

  • Exploit and explore event association: Machine learning, data mining, data system, and artificial techniques are applied here to build event miner tool. This tool aims to exploit and explore event associations that can represent complicated situations happen in real life. Specific models are built based on these associations towards having an ability to detect and predict complex events.

  • Test hypothesis: This stage is very important to get rid of complex event models that do not suit real situations. The social testing field is expected to use for determining which complex event models can be useful for human life.

  • Update knowledge graph: Complex event models that survive after the hypothesis testing stage will be updated to a knowledge graph and be ready for use.

  • Build application: By reusing, transferring, and sharing knowledge graph and APIs. The collaborative development can be leveraged to fast build a new application. In this stage, APIs mashup should be built to share all developed models so people can utilize it to access the knowledge graph as well as integrate different models to build a new model.

Since ESAP shares knowledge and models via APIs mashup, more and more applications are built quickly towards launching into IoAs more and more data. These data will be the new input data of ESAP. Thus, ESAP can feed, learn, and evolve during its life cycle. The APIs also eliminates the need for experts with high-level programming skills. The applications can easily be built by smartly matching suitable ESAPs to the queried ESAP to answer/recommend to users. A prototype of an assistant is shown in Fig. 5.

Fig. 5.
figure 5

A universal prototype of an assistant

Full size image

6 Conclusion

In this paper, we have proposed how usage analytics, heterogeneous sensing, data retrieval and applied machine learning can all be brought into one framework namely ESAP which can pave way for a new paradigm of applications that we term as IoA. We have elaborated on the underlying mechanism how PAs, SAs, and IoAs could be built. Currently we are in the process of developing the ESAP and testing it in various use cases from different business and service sectors such as banking, healthcare, etc. We foresee a lot of challenges in designing the service and personal assistants in such a way that all parties place their trust in the system. Once the system is built there is a need for an empirical study to understand how IoA will be perceived both by end-user and business users in smart city/nation context. In order for this system to be successful proper incentives should be devised for all parties involved in the system.