Keywords

1 Introduction

To foster and facilitate the development of new applications, flexible smart systems and complex architectures are proposed. Specifically, these systems are able to connect different domains from different smart city ecosystems together, avoiding single vertical implementations that limit the scalability of solutions and infrastructure.

Available solutions and systems that are developing in vertical markets at various domains, offer little interoperability and sharing of resources and there is a knowledge gap about developing cross-sector smart city systems. To handle these challenges, Komninos et al. [1] proposed the concept of ‘Connected Intelligence Spaces’ that enables synergies between human, machine, and collective intelligence and assess a universal architecture of high impact smart city projects. These spaces for interconnecting distributed smart city ecosystems can be utilised for information sharing.

Existing decentralised smart city ecosystems can be connected via information sharing pipelines as a horizontal layer that contributes to decreased load, but also offers extensive flexibility by promoting a unified architecture. Adjustments are mandatory for overcoming the underlying fragmentation of smart city ecosystems and for contributing to a more universal architecture scalability. This can promote the decision-making process and conclude to universal access frameworks and architectures that are customised to the citizens’ needs.

Discussions for evaluating the performance of heterogeneous architectures and communication protocols are triggered by the research community, to develop a universal smart infrastructure and a universal architecture of city intelligence categorised in four parts (agglomeration, orchestration, empowerment, and instrumentation according to N. Komninos [2]) that appears within smart cities and enables interoperability across smart city ecosystems.

Numerous challenges though, need to be tackled via the collaboration of all involved stakeholders and real-world implementations are needed to assess the theoretical frameworks and architectures in this nascent area.

2 Approaching Smart City Ecosystems from Legacy to New Initiatives

With the advancement of smart city technologies, 27 billion IoT devices are expected by 2025 [3]. This network of interconnected physical devices and software applications constantly produces and exchanges data over the Internet, and creates clusters and networks. Smart cities do not have a single definition, but they are considered as an abstract multi-layered structure that utilises information and communication technologies, data and analytics to operate efficiently, improve the quality of governance and citizen well-being. From a wider scope, a smart city ecosystem includes people, organisations, businesses, policies and laws cooperating to provide solutions in a domain of activity, such as governance, economy, mobility, healthcare, public safety, environmental sustainability, and others [4].

One of the first initiatives that have been proposed for implementing a smart city ecosystem, suggests that every application, service, and IoT device should comply with specific connectivity criteria and communication protocols. This approach is mainly based on standards, principles and strict software architectures and limits the implementation freedom of developers while restricting the types of devices. Compatibility issues arise and the information distribution channels are restricted. Hence, legacy applications are difficult to maintain as they follow deprecated standards and eventually should be abandoned. New initiatives that tackle the problem of legacy ecosystems should be substituted by new holistic approaches that have no limitations and follow open connectivity standards. Universality will not only reduce the overall cost across software tools and hardware devices (human labour, maintenance and hardware cost) but will also gradually transform legacy applications to open standards solutions that digitally transform the old ecosystem.

The purpose of any newly established ecosystem is to provide a path for the next generation of applications. All stakeholders involved in this ecosystem will be able to operate via open data streams. Due to its highly open design, start-ups, SMEs, and big tech companies can equally develop applications to be connected to the ecosystem despite their dynamics. Application interoperability across smart city ecosystems is another major goal of this initiative. Different types of services are seamlessly interconnected and provide communication solutions at the social, political, and environmental domains. Stakeholders can use data from a variety of different sources focusing on the provided services and not on backend developments which will be orchestrated horizontally.

Contemporary smart city ecosystems need to define open architectures and open standards to support interoperability and interconnection of applications by producing and using data in the form of data streams. The adaptation of new standards ensures interoperability across all services and applications. Moreover, accessibility standards need to be integrated at every level of services, organisations, federations, from design to implementation, to promote inclusion. End-users can benefit from the adoption of the new approach by having access to numerous applications and services customised to their needs and accessibility requirements. Organisations and federations on the other hand can collect and combine data from different sources to make predictions, analysis, and apply machine learning methodologies or any other post-processing of data. The utilisation of cross-platform data will immensely support optimisation techniques that could be applied to well-established domains (public transportation, energy, or water consumption) and affect our daily routines.

3 Data and Surveys

3.1 Architectures Towards Systems Integration

The technological growth of the last decade in terms of available open-source and proprietary solutions for managing large scale software ecosystems, in conjunction with their constantly reduced costs, has led to a blooming transition from monolithic architectures into modular, scalable, and auto-deployable containerised environments. Additionally, there is a continuously increasing trend of moving from on-premises and legacy systems to multi and hybrid cloud-based ecosystems. This migration is applied not only to enterprises and private companies but to local authorities and the public sector. According to Danielsen et al. [5] the transition is due to proven cloud computing benefits that include among others, cost reduction, security, flexibility and scalability, mobility and availability, and infrastructure. The tendency for hybrid cloud computing is highlighted in Gartner’s 2021 overview of the top trend hype cycle for digital government [6]. The same report also identifies a clear focus on solution design to meet the agility demands of governmental organisations through the inclusion of Digital Government Technology Platforms (DGTPs), event stream processing, full life cycle Application Programming Interface (API) management, microservices, and packaged business components.

Undeniably, the IoT concept plays a major role in the context of smart cities and several surveys have been conducted to highlight the importance of open cross-compatible IoT platforms and their underlying architectures. Mineraud et al. [7], identified the technical gaps and differences during the integration and development phase among open-source and proprietary IoT platforms and concluded that although open-source platforms can be expanded more rapidly to cope with the emergence of new technologies, proprietary solutions also tend to adapt on new requirements even at a slower pace. Due to the wide diversity of sensors and their supported systems, there is no global architecture to cover all needs. Still, some universally accepted architectural characteristics should be applied to every proposed solution, such as scalability, high availability, and flexibility. The most commonly accepted architecture for IoT projects consists of three layers, namely; the perception (e.g., how data are produced from sensors and IT systems), the network (e.g., how data are transferred), and the application (e.g., how data are displayed to end-users). MongoDB’s technical article [8] considers the three-layer architecture and proposes a five-layer architecture that stacks to perception, transportation, processing, application, and business layers. Another approach to define an IoT architecture according to several studies is a four-stage workflow consisting of sensors and actuators, Internet gateways and Data Acquisition Systems (DAS), Edge IT, and data-centre and cloud, while some others include a fifth stage which involves user interaction, control, and feedback.

At a lower level, the most widely used IoT standards to support interoperable data exchange between devices and the cloud, are the MQTT, an OASIS standard messaging protocol that offers bi-directional secure communication, and the CoAP, a service layer protocol mainly for use in resource-constrained devices. For larger universal implementations supporting a wider variety of backend applications and services, a common approach is to apply the oneM2M standards that most importantly support a syntactic and semantic interoperability solution via a set of ontologies and XML schemas for connecting cross-silo IoT systems. Thus, it is of high importance that a semantic vocabulary is used across systems to minimise the integration complexity.

Besides the IoT context, a smart city ecosystem is composed of thousands of heterogeneous larger IT systems, each one having its business logic and handling its data. A well-established method to interconnect these systems is each one to acknowledge every other system’s API (tightly coupled). The use of APIs is the most widely used methodology of integration; gateway-service implementation is mostly used in systems without access to source code, and proxy-service implementations based on extended SDK are used at device-level platforms that are frequently inaccessible. Although using APIs, to connect contrasted systems is broadly used, it demands bilaterally agreements between providers and deep knowledge of their available methods (e.g., REST with the use of open API standards). It also requires constant monitoring for possible API’s schema updates, newly introduced parameters, endpoints revisions, and other issues. To overcome these obstacles, an architecture based on the MQTT approach at a much larger scale, is suggested. Every system keeps its existing operations and logic but in addition, it produces a payload of data that streams under one or more topics (aka categories) without necessarily knowing a priori which systems or other smart city ecosystems are going to receive them (loosely coupled). Vice versa, systems willing to receive data from others, just need to know the topic names. To support such architecture, a cluster of messaging brokers, easily scalable, must be included in the ecosystem. The use of data streams over Kafka is a proven solution offering distributed coordination and can support such a large-scale implementation, allowing new kinds of real-time functionalities. Such an approach is adopted by the “Improve My City” application [9] which produces and streams data in kafka topics, which are then consumed by a dashboard. A controlled parking system in Thessaloniki [10] uses kafka streams to fetch data from differerent sources and creates interactive visualisations. Moreover, for newly incoming IT systems into the smart city ecosystem, it is suggested to avoid monolithic designs and use modern cloud computing as-a-service approach such as infrastructure-as-a-service (IaaS), platform-as-a-service (PaaS), and software-as-a-service (SaaS) and also to adopt the microservices architecture for their applications, orchestrated by solutions such as Kubernetes, Mesos, Docker Swarm, OpenShift, and others.

3.2 Platforms and Services

The enforcement of specific communication protocols and connectivity criteria in smart city ecosystems limit the use of IoT devices and software modules and restrict the easy development of niche, reusable and scalable software applications and services. Holistic approaches better engage the ecosystem’s layers and pave the way for universality and interoperability across them. Multi-layered smart city platforms are robust solutions that tackle the heterogeneous nature of IoT devices and seemingly connect them to the smart city backbone. In the early years, smart city platforms mainly included protocols and methodologies for on-demand data aggregation. More sophisticated platforms focus on the distributed nature of the smart cities and make extensive use of APIs for interoperating with hardware and software devices. As a next step, push protocols are used for data retrieval and communication. Data streams and topics are broadcasting data from multiple sources while distributed cloud platforms are collecting them. This enables the implementation of hardware and software agnostic methodologies, and the development of a plethora of services.

A common solution is the implementation of platforms that support numerous APIs and provide a set of libraries for establishing connections to IoT devices and third-party software tools. FIWARE is a widely used platform that offers such APIs for developing web applications and many contemporary solutions are using FIWARE as a building block. Pereira et al. [11] reviewed several smart city platforms regarding their functional requirements, indicatively; data management, application runtime, sensor management, data processing, external data sources, services, tools, city models, distributed sensing, resource discovery, resource, and events management. The authors’ research output SGeoL is a multi-layered smart city platform for handling heterogeneous data. Trilles et al. [12], presented the SmartUJI platform that aggregates university-related data sources and offers them to the public using RESTful APIs and web services. SmartUJI provides the content, the service, and the application layers. Similar is the case of Webinos and CityPulse that enable the development of applications through APIs as well.

Massana et al. [13] proposed a multi-layered framework to monitor activities in the smart city ecosystem. The services layer handles data streams while the application layer provides the dashboards that interoperate with the end-users. The platform Sense Our Environment (SEnviro) follows an inherent IoT and Web of Things (WoT) approach using low-cost, open-hardware and open-software, energetically autonomous and interoperable solutions. At the application layer, a set of web applications are provided. The European Commission also supported the SmartSantander project as a part of the Future Internet Research and Experimentation initiative, for monitoring the environment’s pollution, the parking positions, and irrigation systems. PortoLivingLab has been developed in Porto and has a multi-source sensing infrastructure for data aggregation and management applications.

Middleware solutions are also taken into consideration for tackling heterogeneity both in hardware and software components. The EPIC (European Platform for Intelligent Cities) project proposed a middleware tool for tacking interoperability, extensibility, and reconfigurability in smart cities. Sofia2 is a middleware that enables interoperability between multiple systems and devices, offering a semantic platform that makes real-world information available to smart, mainly IoT-oriented applications. Similarly, Civitas could be used for application development and could tackle the heterogeneity of the smart cities using abstract interfaces.

Cloud-based platforms are connecting IoT devices to the smart city ecosystem. SIGMA and Kaa are cloud-based solutions for storing, handling, processing and presenting IoT data. The SureCity platform is using Azure Cloud services and provides dashboards for smart app development (Pardo-García et al. [14]). Snap4City platform is a cloud IoT solution that focuses on microservices (Badii et al. [15]).

Service-dominant platforms are also proposed to focus on key contributors in a smart city and describe that value can be co-created in the establishment of a platform (Yu et al. [16]). Carriots is a Platform as a Service (PaaS) solution designated for IoT and can be used to connect the information-providing infrastructure to a smart city. InterSCity is an open-source platform that could be also used for the development of microservices.

Agent-based distributed platforms are tackling scalability issues and support the implementation of nodes for establishing a robust network infrastructure and handling the IoT devices. The design and development of distributed smart city IoT platforms for handling large volumes of data is presented while an approach for reusing functionalities of legacy applications is examined. The IoT landscape includes many manufacturers, protocols and communication technologies and current platforms have difficulties transparently supporting them while having scalability issues. An Apache Kafka based platform could tackle scalability and reusability issues using a database, a data streaming, and an application layer (Chamoso et al. [17]).

3.3 Cross-ecosystem IoT Infrastructure

As smart cities continue to grow, the migration of citizens to urban areas has imposed various challenges. Most of the established IoT ecosystems are combining multiple information flows into one single platform and do not fully support integration with third-party applications. This approach results in IoT ecosystems known as silos. To solve this inherent problem that involves multiple proprietary and open IoT platforms, the EU’s ambition is the establishment of Open IoT ecosystems. The US government supports multinational corporations such as Google, Amazon, Facebook and Apple to develop state of the art IoT ecosystems (Miguel et al. [18]). On the other hand, the EU is also funding SMEs for the development of innovative ecosystems that contribute to the growth of a more sustainable smart city ecosystem (Kubler et al. [19]).

The establishment of widely used open IoT platforms imposes challenges from an integration perspective and usually concludes with the development of APIs and web sockets for linking third-party applications. To exchange information with multiple platforms, to support the vertical silos and to create a unified ecosystem that guarantees interoperability across all services, standards are adopted. Ubiquitous connectivity and disruptive innovations in several sectors (e.g., transportation, energy, manufacturing, healthcare, cities, etc.), demand the creation of open IoT ecosystems as sustainable connectivity and information gathering solutions. The design of state-of-the-art platforms should support ad hoc and loosely coupled data flows among hardware devices, software components, data sources, and users.

Cross Platform Interoperability.

Extensive research has been conducted in published papers and funded projects to support cross-platform interoperability. Standards and abstract interfaces enable the connection of IoT devices and software applications to a multi-layered ecosystem that aggregates data streams from various sources. Chaturvedi and Kolbe [20] proposed the use of OGC (Open Geospatial Consortium) standards to address cross-platform interoperability issues. They conducted their study at Queen’s Elizabeth Olympic Park in London and highlighted the advantages of integrating geospatial standards for collecting information from heterogeneous data sources in a semantic architecture approach. Similarly, Bröring et al. [21] state that cross-platform interoperability is critical for avoiding vertical silos. To validate their statements, the BiG-IoT project developed an open-source ecosystem for interoperable communication across multiple IoT platforms based on discovery methodologies, marketplaces for data gathering and monetization schemes.

Although standards for communicating between multiple IoT platforms have been proposed, their adoption rate is significantly low. This is due to the fact that companies use commercial proprietary products which are difficult to be adapted to open ecosystem initiatives and solutions. Standards such as O-MI (Open Messaging Interface) and O-DF (Open Data Format) have been tested on projects such as bIoTope funded by the H2020 Research and Innovation Programme (Javed et al. [22]). In order to assess the potential of the standards, extensive trials were performed in three European cities. The VITAL platform has been proposed as a solution for connecting and integrating diverse data sources using semantic data models (Kazmi et al. [23]). There is evidence that standards can promote cross-platform interoperability.

Open IoT Ecosystems.

The concept of open IoT ecosystems has been proposed recently. Open stands for the ability to support open standards for interoperating with third-party platforms, applications and services. Citizens and companies benefit from the utilisation of data pools that aggregate heterogeneous data sources in the smart city ecosystem (Ahlgren et al. [24]). Robert et al. [25] stated that a scalable open IoT ecosystem should be broken down into multiple layers and each layer should have specific communication and access rights. More importantly, to integrate heterogeneous data sources and avoid the creation of IoT silos, methodologies for discovering, connecting and integrating IoT devices from external platforms must be adopted as means of standardising a communication layer.

Automated discovery and connection methodologies have been tested on a scalable IoT testbed (Javed et al. [26]). The authors implemented O-MI and O-DF standards as the messaging format across the connected devices. The hierarchy was structured based on XML which eliminates unnecessary conflicts from connected IoT devices during integration in a unified schema. Although open standardisation is promising for avoiding IoT platform isolation, software limitations should be addressed for shifting to a more scalable and sustainable approach.

3.4 Data Interoperability and Data Transfer Across Ecosystems

Smart cities follow the system-of-systems architecture of cities and an important challenge in developing smart city solutions across (sub)systems relates to data. In particular, the compilation of data from different sources, the orchestration of data, the use of datasets across city (sub)systems, the re-use of same datasets to support different functionalities, and the use of dataset from one system to develop solutions for another system (Liu et al. [27]; Bischof et al. [28]; Gupta et al. [29]). To discuss these challenges of data interoperability across smart city ecosystems, we refer to three cases and experiments.

The IBM design for an open data system in Thessaloniki was a free consultation offered by the company to the city. Thessaloniki, Greece, was selected through a competitive process as one of 16 cities to be awarded a “Smarter Cities Challenge” grant in 2015–2016. A team of six IBM experts worked in the city for three weeks in collaboration with many stakeholders from universities, the government, and the business community. They delivered recommendations on how to organise an open data system that encourages transparency, benchmarking, performance measurement, and data-sharing between public departments, businesses, universities, non-governmental organisations, and citizens (ΙΒΜ [30]). The findings of this assignment highlighted the fragmented and scattered data among multiple recording systems and departments, incomplete data collection, data storage in different formats, data that is not shareable or readily consumable, inaccurate data because of undefined and non-standardised collection and storage, and unclear data ownership. The IBM assignment concluded with five strategic recommendations to the city administration and stakeholders to develop a collaborative city dashboard.

  • Reorganise IT-related departments to enable open data policies and practices, designate a leader for open data, policy and process and streamline services to create efficiency in open data efforts.

  • Establish an open data strategy and consistent understanding across City departments and stakeholders, managing the coordination between stakeholders

  • Foster an environment that supports collaboration in dashboard development with ideas from technology, academia and business that enable diverse groups to work together

  • Establish a publishing process and maturity model that put open data into practice, increasing the City’s ability to govern and publish data and transparency of City activities

  • Address resource constraints through investments, strategic partnerships, and change management.

Gaia-X is a more complex system, a federated data infrastructure establishing an ecosystem in which data is made available, collated and shared in a trustworthy environment [31]. GAIA-X is not a monolithic organisation or platform but a cloud ecosystem. In the ecosystem, data are not stored centrally but at the source and are shared via semantic interoperability. A key concept to achieve this type of collaboration is the concept of “Data Space”. The term refers to the relationship between trusted partners that apply the same standards and rules for data storage and sharing. Data spaces are created by participants that decide to share data. They can be data providers, users, or intermediary organisations. Each organisation can participate in many data spaces and therefore data spaces are nested and overlapping. To ensure data sovereignty and trust, Gaia-X has developed a reference architecture model, which defines the open data infrastructure and how Gaia-X facilitates interconnection, interoperability, and integration of data spaces (Gaia-X [32]). Existing examples include Gaia-X data spaces in the domains of SMEs and industry 4.0, health, education, energy, mobility, finance, in which many organisations collaborate in data sharing. For instance, in the case of smart homes, Gaia-X is building a platform for organising, orchestrating, and optimising data from smart meters on gas, water and electricity consumption. Gaia-X is a European initiative towards a “sovereign cloud” that would end the dependence of the European economy on large US and Chinese hyperscalers (AWS, Microsoft Azure, Google Cloud, Alibaba, IBM). However, there are concerns about whether Gaia-X will achieve this ambitious objective and develop a sustainable business model taking into account the real needs of the market (Autolitano and Pawlowska [33]).

An advanced case of data sharing in Europe is the Open Research Data Pilot. It was launched by the European Commission in the framework of Horizon 2020 as a pilot for open access to research data and improving the re-use of research data across all thematic areas of H2020. The pilot adopts the FAIR data principles (Findable, Accessible, Interoperable, and Reusable) assisting humans and machines in their discovery of, access to, integration and analysis of data associated algorithms and workflows. Findable data relate to metadata, which is registered or indexed in a searchable resource, specify the data identifier as a globally unique and eternally persistent identifier. Accessible data are retrievable by their identifier using a standardised communications protocol, which is open, free, and universally implementable, and allows for an authentication and authorisation procedure. Interoperable data are those using a formal, accessible, shared, and broadly applicable language for knowledge representation. Finally, reusable data have a plurality of accurate and relevant attributes, are released with a clear and accessible data usage license, and meet domain-relevant community standards (Wilkinson et al. [34]). In this direction, a literature review of academic articles published between 2016 and 2019 on the use of FAIR Guiding Principles is presented by van Reisen et al. [35].

The three cases, we summary presented, shows that data reuse and interoperability across smart city ecosystems rely on three pillars. First, on agreements for collaboration between data providers and users belonging to different ecosystems or sectors of activity, establishing partnerships, data sharing strategies, and data spaces for collaboration. Second, on adopting open (“as open as possible, as closed as necessary”) and FAIR principles of data organisation and semantic annotation enabling data sharing and re-use. Third, by appropriate Human-Computer-Interaction, using semantic technologies and identifiers, formal languages for data representation, and rich and searchable metadata.

4 Discussion

Smart cities involve diverse software applications and hardware devices that constitute complex architectures in multiple knowledge domains and according to N. Komninos [36], along with big data and social media analytics and civic technologies, these architectures and their supported technologies allow the creation of smart ecosystems in which connected intelligence emerges. The establishment of a universal access schema across the smart city ecosystem necessitates the adoption of open standards on the communication and integration layer while providing a trustworthy infrastructure for data acquisition and management. Although technologies for connecting diverse IoT ecosystems have been proposed and utilised, third-party applications are still striving to identify and adopt efficient standards that minimise the total development time for building connectors and middleware software tools.

To support the connectivity of ecosystems, data streams and data spaces are continuously under study and new hardware and software solutions are proposed. It is critical to have a consensus on the conceptual design and planning scheme and explicitly propose next generation universal access services. This study analyses four echelons for promoting universal access to smart city data providers: (i) architecture; (ii) platform and services; (iii) IoT infrastructure; and (iv) data interoperability and transfer. At the architecture level, there is a trend for adopting multi-layered cloud-based systems that support both microservices and classic infrastructure, platform and software-as-a-service solutions. This is also the case for platforms and services as distributed multi-layered cloud platforms are proposed for supporting both developers and end-users with services. The IoT infrastructure is focused on providing hardware and software integration middleware tools for avoiding the creation of IoT silos and unify heterogeneous devices and services in a scalable, hardware and software agnostic communication layer. Finally, data interoperability and data transfer rely on the adoption of (i) common strategies at the data provider level; (ii) open and FAIR principles; and (iii) effective and efficient Human-Computer-Interaction interfaces.

Eventually, the collaboration of all the aforementioned technologies that compose a smart city, such as; software platforms, system architectures, IoT, but also, social media, data science, and lately blockchain (as used in the context of intelligent cities [37]), are actually constituting the algorithmic logic under which they operate. In the book “Smart Cities in the Post-algorithmic Era”, the editors [38] conclude that the algorithmic logic should be combined with creativity, innovation and collective and collaborative intelligence in order to be efficient and effective.

Smart city stakeholders should elaborate on the reorganisation of IT processes and establish consistent open data strategies. Heterogeneous data streams should be handled by a multi-layered ecosystem that provides focused services to the citizens with the use of state-of-the-art flexible dashboards. As a last step, there is a continuous need for investments that could be fostered by strategic partnerships between the public and the private domain.

5 Conclusions

The HCI community is examining universality and interoperability in order to provide global protocols, standards and methodologies. This schema expands the usage levels of heterogeneous devices and data sources and supports the decision making process. The cross-ecosystem area of IoT technologies is expected to transform the well-established IoT platforms into a unified ecosystem minimising the heterogeneity while providing the building blocks for future ecosystems. Specifically, initiatives that address the importance of the ubiquitous connectivity and interoperability of the underlying ecosystems are the first steps to a futureproof interconnected ecosystem. Standardisation across all IoT layers, from the data models to the application connectivity layers are crucial for enabling future technologies to be built upon. This will enable the development of new ecosystems that prohibit the formation the vertical silos. Although, innovative projects are already being funded, we are far from concluding to universality and interoperability standards. Future research and experimentation are mandatory to assess the importance and added benefits of standardisation. Messaging, connectivity standards and data provisioning are the underlying pillars of the aforementioned initiatives, that will shape the future of cross-ecosystems interoperability.

The multi-layered approach is considered as a viable solution for aggregating heter-ogeneous data sources in the global smart city ecosystem. IoT agnostic solutions are needed in a global landscape that should embrasse a very high number of billions of software applications and hardware devices that connect to the physical and social space of cities and form complex global ecosystems. At the same time, the platforms are evolving and provide distributed cloud services based on multi layered architectures.