Keywords

These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

1 Introduction

The widespread adoption of sensor networks, actuators and computational resources capable of interacting with people is transforming urban environments [6]. Citizens will have the possibility of being continuously connected in a situation-aware and socially-aware way, both with each other and with the entities around, e.g., typically via some situation-aware social networking infrastructure [28]. This will eventually contribute to define a dense ecosystem whose individual components will enable collaboration between ICT devices and humans, towards the realization of advanced urban services. Such services can contribute to the smart city vision along several dimensions from intelligent transportation systems, to environmental sustainability and participatory governance [15, 24]. Even more, it has been envisioned how they could radically transform urban environments into socio-technical urban superorganisms [12, 28].

Future pervasive urban services, rather than being limited to sensing what happens in the city (as most current approaches do) will leverage on the complementary sensing, computing, and actuating capabilities of humans and ICT devices.

The urban system as a whole will be able to: (a) combine a wide range of information sources (e.g., environmental data from sensor networks, mobility data and social network posts) and sense the current state of the city. (b) perform advanced reasoning on the data to extract patterns and routines taking place. (c) engage in large-scale coordinated tasks to achieve specific goals (e.g., optimize traffic flow in the city, make it more environmentally sustainable, etc.).

That is, the overall urban environment will act as a single superorganism made up of individual organisms (humans and ICT devices) and capable of directing its behaviour towards the achievement of specific urban-level goals.

Engineering collaborative and coordinated services capable of harnessing human and ICT capabilities at large scales challenges current engineering practices and middleware architectures. In this context, the contribution of this work is twofold:

  • It sketches the key concepts of urban superorganisms and of the collective awareness mechanisms at the basis.

  • It outlines the key research challenges to be faced to realize such kind of systems

  • It proposes a self-aware middleware architecture conceived around the goal of tackling the identified challenges.

The paper is organized according to the above points. Section 2 presents our vision on urban superorganism and on the associated collective awareness. Section 3 discusses key research challenges. Section 4 proposes a general-purpose middleware architecture addressing some of those challenges. Section 5 provides a technical overview of the same architecture presenting also some initial experiments we conducted to analyze the feasibility of the architecture. Finally, Sect. 6 provides some concluding remarks.

2 The Urban Superorganism

Fig. 1
figure 1

Collaborative sensing, awareness and action among humans and ICT systems can be put at work in urban superorganisms in the form of a closed feedback loop

Full size image

In future ubiquitous computing scenarios, the very large number of inter-connected entities that can be found in urban environments, whether humans or ICT devices, can potentially be exploited to create what has been defined as a superorganism [12]. In particular, closing the sensing, computing, and actuating capabilities in a loop (see Fig. 1), and making such activities collaborative ones, it is possible to realize coherent collective behaviours, as it is observed in many natural situations, e.g., in ant colonies [4].

A single ant has very limited, local sensing and actuating capabilities, and little or no cognitive abilities. Yet, ants can indirectly coordinate their movements and activities, via spreading and sensing of pheromones in the environment, so as to exhibit, as a colony, very powerful capabilities of sensing (finding food in the environment), computing (finding the shortest path from food back to nest), and action (carrying large amounts of food in the nest). These capabilities make the whole colony seemingly intelligent and certainly adaptive in its foraging activities.

Research in self-adaptive and self-organizing systems have focused on defining a catalogue of bio-inspired mechanisms, with the intent to overcome the limit of ad-hoc implementations that prevent their systemic reuse [7]. Thus, the basic idea is that of providing the bio-inspired self-organizing pattern modules with a set of reusable patterns that could be used to ease engineering of artificial and collective behaviors of urban superorganisms. It is probable that the complementary capabilities of humans and ICT devices and the ability to coordinate and organize them could overcome current approaches and promote collective awareness and complex behaviors.

More in detail, Fig. 1 illustrates the feedback cycle creating collective awareness in the super organism. Advanced finalized and coordinated activities are the result of: Sensing activities in which users supported by ICT devices and services get information about the current state of the environment (e.g., people location data). Understanding activities in which advanced forms of context information are derived from the sensed data (e.g., citizens mobility patterns are identified from the collected location data). Acting: goal-directed coordinated tasks supported by the extracted information (e.g., traffic management on the basis of the identified mobility patterns, car sharing on the basis of people mobility routines, etc.). The results of the activities being performed are then sensed again closing the feedback cycle.

Previous works in opportunistic and participatory sensing have tried to involve users by making use of their devices as sensors [14, 21]. On the opposite side, other works try to detect events or situations by observing users activities on online social networks [23]. However, these works lack a general and unified vision and do not completely tackle the complexity of the global scenario, i.e. they do not explore all the possible convergence of humans and ICT devices. Moreover, they do not fully make use of the very large number of inter-connected individuals and their complementary capabilities to realize a collective awareness.

Recent works are addressing the “smart city” scenario [5]. Current approaches to smart cities (e.g., as those that are carried on by IBM [15]) are mostly related to the “sensing and understanding” facets of the urban superorganism scenario (cfr. Fig. 1), i.e., collecting (typically in a centralized way) data about various aspects of a city life and get a meaning out of it, for the sake of driving decision makers in planning future urban infrastructures. The acting aspect, i.e., the possibility of dynamically involving citizens and ICT actuators to dynamically influence the city dynamics is mostly disregarded. Only a few studies in this direction have been performed, and mostly oriented to steer crowd via mobile phones (e.g., the Tag my Lagoon Project in VeniceFootnote 1), or at directing traffic towards zones with available parking space (e.g., the Santander Smart City ProjectFootnote 2).

2.1 The Role of Humans and ICT Systems

In this section we provide more details on the sensing, computing and actuating activities to be integrated for the sake of enabling the above described collective awareness feedback cycle. People are increasingly equipped with smart phones with powerful capacities in terms of battery life, sensing, computational power and connectivity. At the same time, autonomous ICT infrastructures (sensor networks, security cameras, robots, etc.) are likely to pervade cities in the near future. Accordingly, the future urban environment is becoming a sort of very dense digital ecosystem (Fig. 2).

Fig. 2
figure 2

The table summarises sensing, computing, and actuating capabilities of both humans and ICT devices that could mutually interact within an urban superorganism

Full size image

The components that are going to pervade urban environments are characterized by heterogeneous and complementary sensing, computing, and actuating capabilities, that can cooperate in a goal-directed way. In particular:

Sensing

  • ICT Side. The capabilities in sensing from the ICT side are provided by (a) mobile phones equipped with GPS, accelerometers and cameras; (b) sensors networks and smart objects that follow the Internet of Things paradigm [1]; (c) tags that exploit the wireless short-range communication technologies (NFC, RFID and Bluetooth).

  • Human Side. From the human side, the five senses of humans, which in many situations can supply and be more accurate than ICT sensors, can be put at work for the community, due to the possibility of continuous access to social networks. In addition, users can make available via social networks any other information, thus acting as sorts of social sensors [22].

Understanding

  • ICT Side. The capabilities in computing from the ICT side makes it possible to collect and digest very large amounts of urban data in a short time, and to perform some limited pattern analysis on such data.

  • Human Side. From the human side, on the other hand, one can exploit the capability of recognizing complex situations and patterns (so called human computation [27]), which machines can hardly tackle.

Acting

  • ICT Side. The capabilities in actuating from ICT side can be provided by (a) traffic controllers supporting pervasive solutions in the mobility dimension; (b) public displays that will be exploited to promote adaptable citizens behavioural steering; (c) all kinds of actuators related to critical infrastructures (e.g., energy grid).

  • Human Side. From the human side the key actuating element involved is the user himself, which can perform a variety of actions related to moving or moving items around or changing the properties of some physical entities. In other words, citizens could accomplish actions, by realizing an impact on the environment.

The goal-directed integration of the above capabilities and activities, will allow to close the collective-awareness feedback loop enabling large-scale coordinated behavior among humans and ICT devices and services.

2.2 Application Scenarios

In this section we present some exemplary application scenarios that could be enabled by the vision of the super organism and by the above defined collective awareness cycle.

Mobility. Among many capabilities that future urban superorganisms will exhibit, the first that we expect to be in place, and for which we already observe embryonic examples around, will relate to urban mobility [10, 13, 25]. Specifically, it will relate to the capability of affecting (i.e., steering) the movement of vehicles or pedestrians, and thus improving the overall efficiency of urban mobility while reducing the stress of users, due the improvement of traffic flows and the avoidance of traffic congestion. A variety of sensors already exist to detect the conditions of traffic or crowd in urban environments. In addition, users are increasingly given the possibility to contribute to such sensing activities by posting information on social networks or by opening access to their navigators and smart phone sensors. All this information can be used to understand how to improve traffic flows or how to avoid congestion. To this end: actuators such as traffic lights and digital traffic signs can be put at work for vehicles; public (wall mounted) and private (smart phone) displays can be exploited to suggest directions to pedestrian. However, one could push the capabilities of superorganisms much further. For instance, one can think of dynamically matching the similarity of planned vehicle routes and of merchandise to be delivered to dynamically self-organize a very flexible ride sharing and shipment services. In general, urban superorganisms induce a change in the dominant paradigm for the provisioning of mobility services: from sensing mobility patterns and adapt existing services to them, to dynamically collect mobility needs and self-organize the role and mobility patterns of vehicles accordingly.

Sustainability. As an additional example of how the capabilities of future urban superorganisms can impact urban life, just imagine sensing in real-time information related to energy consumption, to compute sorts of instantaneous urban carbon footprints for specific areas of the city or for specific groups of citizens, other than for the city as a whole. Public displays can then be exploited to share this information and possibly some analysis of the factors contributing to it, and personal displays can be possible exploited to let individuals and groups to become aware of their own contributions to the urban carbon footprint. On these bases, one could think of steering the behaviour of individual citizens towards more energy efficient behaviours, or at engaging groups of citizens in self-organized collaborative actions aimed at solving/improving specific energy problems in specific urban areas to supply the lack of actuators suitable to the purpose (e.g., detecting open windows and closing them).

Taking Care. Via similar means, it could be possible to dynamically involve citizens in proactively helping to taking care of the city, e.g., to help keeping it city cleaner or making it a safer place for everyone. For instance, one can think of dynamically engaging people to temporarily take care of (or simply take a look at for some minutes) children on their way to school, whenever the current activity and known habits of some persons suggests. Ideally, in the presence of enough matching persons willing to be involved, and possibly complementing sensors (e.g., cameras) and actuators (e.g., robots) already in place for that purpose, one can make sure that the whole path from home to school of every children in a city is properly covered and taken care of.

Feeling Part of It. Beside thinking at measurably useful objectives and services for which urban superorganisms can be put at work, their advantages could also be in the (not easily measurable) way by which they will improve our experience of living in urban environments. In particular, acting and moving around in a city by being given feedbacks on the effect of our own existence in it, and by making possible to observe ourselves in relation with our environment and with the other citizens, can make most of our everyday actions inherently more pleasant and rewarding, and can promote a renewed and stronger sense of citizenship.

In addition to these exemplary applications, we think that in the next future innovative collaborative and collective behaviours, expressing various forms of urban awareness and intelligence, will take place. These will dramatically change the way we move, live, and work, in our urban environments.

3 Challenges for Superorganism Architectures

The vision of the urban superorganism presents a number of challenges that can be hardly dealt with by present middleware architectures. In this section we present a number of such challenges, while in the next one, we present a middleware proposal addressing them.

3.1 Heterogeneity and Interoperability

The software architecture has to provide an abstraction layer on top of different individuals, both humans and ICT devices, by adapting their heterogeneous and complementary sensing, actuating and computing capabilities [9, 19]. The complementary use of sensing and understanding capabilities of humans and ICT devices has to lead the coordinated learning process towards reconstructing a collective awareness of the state of the city. A common example that fits this situation is expressed by the capturing of pictures of a traffic jam both from users cameras and Closed Circuit TeleVision (CCTV) cameras, to make sure that all the images are properly tagged with information automatically generated by devices (e.g., amount of vehicles involved) and further enriched by humans (e.g., the reason they are stuck in traffic), due to their higher classification capabilities. The challenge is to realize an abstraction layer able to continuously observe the superorganism status and plan strategies to reach specific goals by making use of heterogeneous individuals with evolving sensing, actuating and computing capabilities.

3.2 Dynamic Re-configurability

As emerged by the heterogeneity challenge, the software architecture should show a certain degree of flexibility. Thus, the ability to dynamically execute heterogeneous code with heterogeneous sensing, actuating and computing capabilities is a key challenge to be tackled. For instance, in the case of an accident report executed by a citizen, the architecture has to support a service that requires the user interaction and thus has a user interface; while, in the case of the same task executed by a remote camera, the latter has to support services exploiting the sensing, actuating and computing capabilities of devices. Furthermore, dynamic service composition and re-configuration is needed. The design of an architecture dealing with these challenges will make applications interoperable and able to self-reconfigure and self-optimize depending on the execution context.

3.3 Interconnection

As emerged by the case study, individuals have to be connected and able to exchange messages for both (a) supporting collective behaviors (e.g. the coordination effort required to steer individuals to take photos of a road intersection), and (b) gathering individual awareness to infer complex situations that involve the superorganism rather than specific individuals. It is worth noticing how this requirement calls for innovative data fusion techniques. In fact, at both the individual and superorganism levels multiple data streams of information sources have to be processed and put together to build up a coherent picture of operating conditions.

3.4 Behavioural Steering

The effectiveness of the infrastructure supporting the urban superorganism is determined by the amount of people actually involved in the collective sensing, computing and actuating phases. So, from a social perspective, it depends mainly on how deep individuals are steered by collaborative behaviors. Academic literature present several studies on modeling and evaluating behavioral changes driven by ICT devices [16, 17, 29]. As precisely described by Klein et al. [16], behavioral changes can be supported by taking a closer look at underlying determinants of behavior change, focusing on how users can be persuaded to establish a desired behavior. This practice results in the identification of behavioral patterns composed by a sequence of behavior determinants, which are solicited through proper HCI techniques (e.g., provide the user with automated reminders, valuable suggestions, and tailored feedback on her activity), and then evaluated with a related computational model based on theoretical frameworks of behavior change, to classify the degree of individuals motivation, awareness, and commitment to adopt new desired habits. However, a satisfying evaluation of behavioral change models effectiveness needs to be protracted for years, in order to distinguish short-lived changes from permanent ones.

3.5 Dynamic Selection

One should evaluate which individuals are more suitable to be involved by taking in account different constraints. For instance, in the case study, many CCTV cameras and humans could be available at the same time to capture traffic information to feed any Intelligent Transportation System. Strategies taking into account different constraints (e.g. geographical areas, individuals status and sensing, actuating and computing capabilities) are needed to pick up the most affine individuals for the desired behavior. These strategies could be based both on explicit interactions (e.g., sending a message to an individual) or implicit interactions (e.g., sub-sampling the whole population of individuals satisfying specific constraints).

3.6 Context Awareness

As technologies evolve, new types of sensors become available. Chemical, electric, optical, proximity and position sensors can provide data about environment, weather, presence or movements of and between different entities part of the city life. Furthermore humans can also act as a type of social sensor through social networks or their mobile phone signals [9, 19]. These data sources, made of humans and ICT devices, will produce, continuous streams that will generate a very big data set, specially, if we consider the temporal dimension of data. From a computation concern, the issue is about finding suitable pattern analysis algorithms to extract high-level knowledge from sensed data [2]. As the number of available data sources and algorithms to process them is constantly increasing, the perception is that the algorithms to extract relevant information from data are already there. The important challenge is to find ways of combining them together so that results coming from one data can validate and further describe results from other data [3, 11, 18].

3.7 Mixing Bottom-Up and Top-Down Design

Designing with a top-down approach means that all the requirements of a software architecture have to be taken into account first; systems engineered in this way have a predictable and measurable behavior but are not capable of coping with dynamic execution-context; while systems designed with a bottom-up approach are more robust and suitable for a pervasive environment but predicting their behavior and controlling them by design is not an easy task. In the design of architectures for urban superorganisms, both of the two approaches are needed and finding and tuning the optimal trade-off between them is a key challenge to be tackled [6, 26].

4 Architecture

As a first step to create the basis for the collective awareness schema and to address some of the challenges presented in the previous section, we designed and developed a middleware architecture supporting collective sensing, understanding and actuating capabilities. Such an architecture constitutes a general-purpose awareness framework that could be used as a starting point for many superorganism services.

4.1 Conceptual Viewpoint

Fig. 3
figure 3

The architecture seen from both conceptual and engineering viewpoints. From the conceptual viewpoint, it is structured around three layers, namely sensor, classifier and awareness layer. From an engineering viewpoint, the architecture is implemented on the top of industrial-level Java technologies. Each module is actually an OSGi component enriched with iPOJO and Apache Camel functionalities (right). The skeleton of a simple component is also reported (left)

Full size image

The architecture (see Fig. 3-right) is structured around four layers, namely sensor, classifier, awareness and actuator layer. Each layer can host multiple modules connected to each other via application-definable topologies. The data flow from sensors (i.e., both hardware and software) trough the whole architecture by means of in-memory queues enabling modules decoupling and many-to-many asynchronous communications. Each layer can host multiple modules (i.e, sensors, classifiers, awareness modules, actuators and queues).

The sensor layer hosts modules that are in charge of retrieving raw data from physical sensors and of preprocessing them. An example could be a module acquiring images from a camera and cropping and resizing them. Other examples could be modules acquiring facts from social networks, such as Twitter, Facebook or Foursquare. At the time of writing, we have already implemented modules for reading data from Android devices. In general, this layer addresses the heterogeneity and interoperability challenge (Sect. 3.1) in that it provides a uniform access to the various sensors and devices.

The classification layer hosts modules that consume data coming from the sensor layer and classify (i.e., generate semantically richer information) them. An example could be a module able to classify the activity performed by a user by processing accelerometer data. At the time of writing, we have implemented modules for classifying user activity, location, speed, vehicle used on the basis on common smartphone sensors. It is worth noting that our goal is to build a general-purpose awareness framework that could be used as common basis for both research and application development, not to solve every possible classification problem. Specific applications will need their own modules to be developed. This layer tackles most of the context awareness (Sect. 3.6) challenges in that it provides a wide range of context-modules to be integrated to fulfil the application needs.

The awareness layer hosts modules consuming labels produced in the classification layer and feeding external applications with situational information. These modules might have different goals depending on the application. However, they could be divided into two main classes. The former comprises modules delegated to sensor fusion processes. These modules receive labels, eventually conflicting, coming from multiple classification modules and apply algorithms to achieve higher semantic levels. The latter, instead, is related with the capability of the framework of monitoring and controlling itself. In a sense, the awareness layer could be the key for building an awareness module that is aware of itself. For example, it would be possible to integrate within this level modules observing the internal status of the framework and activating different classifiers and sensors depending on the operating conditions. This capability could be used to achieve both improved classification accuracies and reduced power consumption levels by continuously selecting the most suitable classifiers and sensors. The strategies used to select sensors and classifiers might change depending on the application. This layer tackles both the context awareness (Sect. 3.6), and dynamic selection (Sect. 3.5) challenges in that it allows to fuse information together and to choose the most valuable information providers.

The actuator layer hosts modules to enact specific activities and to steer the behavior of users toward specific goals. On the one hand, the actuator layers contains modules to drive the actuators of physical sensors (e.g., to control smart household devices). On the other hand, it contains visualization and user-interaction modules to ask/steer the users to specific actions. At the time of writing, we are implementing modules for user interaction for Android devices. In the long term, the idea is to incorporate in this layer incentivization modules [20] and mechanisms taken from persuasion theory [8]. The use of these latter techniques addresses the behavioral steering (Sect. 3.4) challenges.

4.2 Engineering Viewpoint

From an engineering viewpoint, the architecture is implemented on top of industrial-level Java technologies (see Fig. 3-left). Each module is actually an OSGi component. Because of this, modules (i.e, components) within this architecture can be plugged, removed and reconfigured at runtime. These capabilities, related with the adoption of a Service Oriented Component approach are crucial to address the dynamic reconfigurability (Sect. 3.2) and interconnection (Sect. 3.3) challenges.

On top of OSGi, we have an iPOJO layer. iPOJO is a container-based framework handling the lifecycle of Plain Old Java Objects (POJOs) and supporting management facilities like dynamic dependency handling, component reconfiguration, component factory, and introspection. Moreover, the iPOJO container is easily extensible and allows pluggable handlers, typically for the management of non-functional aspects.

On top of the iPOJO framework we build the support for the staged and layered architecture by making use of Apache Camel. This framework provides components with the capability of asynchronously processing data streams and communicate through in-memory queues. These queues allow modules belonging to different layers to continuously communicate each other with minimum hardware requirements. Considering that pattern classification and analysis has a central role in situation awareness, we wrapped well-know data manipulation libraries within the framework. For instance, experiments presented in this paper made use of Weka and jMIR.

Overall, the proposed architecture allows developers to select the required modules, define the topology of data flows and specify their reconfiguration strategies. The middleware takes care of connecting all the modules and to reconfigure them whenever needed. Such high-level support will allow developers to focus on software engineering-level tasks such as those emphasized in Sect. 3.7 about mixing top-down and bottom-up approaches.

Fig. 4
figure 4

We have evaluated memory consumption and throughput of the awareness module by dynamically increasing the number of parallel components in the classifier layer that process a data stream without further computations

Full size image

5 Experimental Results

To quantitatively assess and validate the feasibility, in terms of performance, of exploiting our awareness module in a superorganism architecture, we have deployed and tested it on servers with Core Duo 2 CPUs operating at 2.2GHz and 5GB of RAM. In the tests we used MacOSX 10.6 with JVM v1.6. The framework is running on Apache Karaf 2.3.0 OSGi container. To validate our implementation, we have stressed the awareness module by realizing a sensor that produces a high traffic load of 10,000 messages (with small payload). As experimental evaluation, we have estimated the performance by dynamically increasing the number of parallel components in the classifier layer that process this data stream without further computations. By dynamically increasing the number of components, we increase the number of messages that have to be processed in the entire module, because the classifiers subscribe the same data stream. The first metric used to evaluate the performance is the memory usage. The corresponding experimental results are reported in Fig. 4. In particular, the figure show how the heap memory usage increases linearly with the number of messages processed.

The second metric used to evaluate the performance of the awareness module is the throughput, in terms of average number of messages per second processed by components in the classifier layer. The corresponding experimental results are reported in Fig. 4. In particular, the graph shows how the average throughput of the system decreases linearly with the increase of parallel components that consume the data stream. However, in the worst case represented by eighty parallel components in execution in the classifier layer, the average throughput remain acceptable, by considering that all the components are able to process 411 messages per second. Considering that this test has been run a single machine, we believe that this would be enough to handle the large majority of circumstances.

These results demonstrate how (a) the bottleneck in the awareness module is the CPU that limits the scalability in terms of number of messages processed per second; (b) the overhead introduced by a self-reconfigurable and highly adaptable awareness module is negligible, because the performance decreases linearly with an increasing load and number of components.

6 Conclusion and Future Works

As we have discussed in this paper, we believe that it will be possible to exploit socio-technical superorganisms to deliver complex collective urban-level services. In our opinion, innovative collaborative collective behaviours expressing various forms of urban awareness and intelligence will take place, and dramatically change the way we move, live, and work, in our urban environments. However, to reach this goal, many research challenges need to be addressed, and suitable middleware infrastructures have to be developed. At the time of writing, we are in the process of completing a first prototype implementation of the proposed architecture. In addition, our future work includes testing the infrastructure in controlled (campus-level) environment and, later on, start experiencing it in real-world situations with simple urban awareness services.