Keywords

1 Introduction

The modern trend of increase of efficiency and safety of the vital processes in the urban environment is the introduction of new information systems and technologies in the form of smart environments [1]. Currently, there is a Smart & Safe City concept [2] in terms of the project’s implementation such as Smart Manufacturing, Smart Houses, Smart Light, Smart Energy, Intelligent Transportation System, Smart Road etc. This concept involves the development of an intelligent information and computing environment for solving the problems of distributed monitoring and support of municipal management processes with the necessary level of life activity [3]. The smart environment will combine various factors for the development of urban area elements into a single system to ensure universal access to information, computing and telecommunication resources. The main direction is the development and implementation of smart big data processing technologies, which are collected from distributed sensors, measuring devices and automation devices in the Internet of Things (IoT) [4]. The result of the synthesis of Smart & Safe City intelligent environment should be to ensure maximum comfort and safety of human life in the urban infrastructure and highly efficient production in the industrial sector.

The paradigm of an intelligent multimodal environment includes three basic concepts: (a) Ubiquitous (Pervasive) Computing and Networking [5]; (b) Intellectual Assistant (Ambient Intelligence) [6]; (c) Smart Environments [7]. Synthesis of the intellectual multimodal environment includes wide introduction and application of technologies:

  • remote control of objects and processes in automatic mode in real time,

  • wireless sensor data exchange,

  • secure collection of information through sensor networks,

  • intellectualization of sensors, automation devices, home appliances, etc.,

  • multi-agent big data processing,

  • predictive modeling of situations for decision-making,

  • convergence of information, computing and telecommunication technologies,

  • standardization of inter-machine interactions between components of the intellectual environment, etc.

The key components of the intellectual environment are:

  1. 1.

    Intelligent sensors (sensors, measuring devices, photo, and video fixation devices) capable of by the built microcontroller to data processing, change modes and actuate with monitoring object.

  2. 2.

    Telecommunication networks of broadband data transmission (fiber-optic and wireless) and mobile communication systems.

  3. 3.

    Satellite navigation systems.

2 Intelligent Transportation Systems and Smart Road

In the Smart & Safe City concept, the important task is to solve the problem related to the development of intelligent transport systems (ITS) and intellectual environment Smart Road [8, 9]. The terms mean the presence of built-in intellectual functionality in vehicles and on objects of the road and transport infrastructure, as well as an intelligent monitoring system for the purpose of automated traffic management and security. In fact, ITS is a system, which uses innovations in the transport system modeling [10] and traffic management, providing end customers informative and safety. The level of interaction between participants (ITS) in the movement growing compared to conventional transport systems.

ITS differ in their technology - from simple car navigation systems, regulation of traffic lights, systems for regulating cargo transportation, information placard systems, car number recognition systems and vehicle speed registration to video surveillance systems, parking management, weather services, etc.

The main classes of ITS are:

  1. (1)

    In-vehicle ITS. Includes sensors, microprocessor cards, and displays, which provide drivers geospatial information on traffic situations.

  2. (2)

    Infrastructure-based ITS. They present various information, including meteorological information, by the roadside information panels, as well as control the traffic flow.

  3. (3)

    Cooperative ITS. Inter-machine interaction is carried out between road infrastructure objects and vehicles (vehicle to infrastructure, V2I), between vehicles (vehicle to vehicle, V2V).

The Smart Road intellectual environment are handled large amounts of data in real time, the bulk of which are photos and video streams from surveillance cameras and photoradar systems. The aim of Smart Road component is to influence the behavior of cars and drivers to optimize the use of transport routes and vehicles, to prevent abnormal and emergency situations and to improve road safety.

Three types of factors affect the risks of road accidents:

  1. (1)

    Anthropogenic factors, which can be caused by changes in the behavior of drivers and pedestrians, speed of movement, health status at a particular moment, etc.

  2. (2)

    Technogenic factors, which are determined by possible problems with vehicles in the process of movement.

  3. (3)

    Infrastructural factors, which are dependent on changes in the state of road transport infrastructure.

Smart Road components need to reduce risks promptly perform such monitoring tasks:

  • monitoring of the road surface state,

  • meteorological monitoring,

  • monitoring of traffic flows and speeds of transport units,

  • monitoring violations of traffic rules,

  • rapid detection of traffic accidents.

Monitoring results should be transmitted to road users through traffic channels, cellular, trunk and satellite radio communication channels, information boards and dynamic road signs.

The main components of the Smart Road environment are:

  • Intelligent monitoring system (IMS) for traffic management,

  • a system of road users warning,

  • feedback system to take into account social reactions of road users and other people,

  • intelligent traffic lights and other signaling systems,

  • intelligent video surveillance cameras and photo and video-fixing complexes of road accidents,

  • satellite systems of transport monitoring,

  • intelligent parking and loading areas,

  • road sensor systems for unmanned vehicles,

  • cars with built-in intelligent transport system,

  • electronic payment systems for road services, etc.

In Russia, the development of ITS is now at the stage of active implementation of primary elements, such as vehicle monitoring system based on GLONASS [11], emergency care system drivers ERA-GLONASS, dynamic maps of public transport, etc. Currently, there are no ready commercial solutions in the field of communication systems between machines (V2V) and communication systems between machines and intellectual infrastructure (V2I).

3 Convergent Model for Processing Big Sensor Data in Smart Road Environment

For the interaction of the components of Smart Road, a transportable wireless environment is required in the form of a segment of the Internet of Things [12], which is designed to provide information exchange of complexes, data center servers, ITS, aerial surveillance systems, mobile users.

For interaction, Smart Road component a transport wireless environment is required in the form of an Internet of things network segment, which is designed to provide information exchange complexes, servers, data centers, ITS, air surveillance systems, mobile communications. Such a medium is heterogeneous, as it is realized through various technologies: wireless sensor networks, cellular networks, WiFi networks, satellite navigation systems. In this environment, a secure data exchange mechanism is implemented. The heterogeneous transport environment for the collection and processing of photo and video data is realized within the framework of the convergent model of computing and data storage [13].

Convergence is defined as the interlinking of computing and storage technologies such as media, content and communication networks that have arisen as a result of evolution and popularization of the internet as well as activities, products, and services in the digital media space. Convergence also refers to the phenomena of a group of technologies which are developed for one use but is being utilized in many different contexts.

Convergence of technology happens due to the digitization of content. The content can be text, graphics, video, animation, and audio. Digital technology allows the use of multiple communication modes in a single network. The reason of convergence is the rapid advancement in the field of the internet along with the emergence of various products and services, the availability of carrier technology with high bandwidth.

Term convergence of networks means telecommunications technology convergence process with the appearance of similar characteristics in network equipment, communication channels, network standards and protocols, data transfer processes [14]. The result of convergence in this aspect is the synthesis and development of a heterogeneous transport environment with the integration of a variety of information and computing services for the big sensor data processing. An example of a convergent platform for collecting and transmitting sensory data is the Internet of things built into the Internet.

The convergence of distributed computing and storage technologies can be used to implement processing big sensor data from photoradar complexes and video cameras.

Four models of distributed computing can be distinguished: GRID computing; cloud computing; fog computing, mobile computing. GRID calculations are based on the architecture of computer networks of the individual compute nodes (Fig. 1).

Fig. 1.
figure 1

The data flow diagram of convergence computing model

Full size image

Computing process provides for a distribution separate parts for the task to currently free network computing resources. This approach is used for tasks too complex for a single node. Cloud computing - is not only the allocation of tasks on the network nodes of computing resources. This model is used for the ubiquitous network access to a common pool of configurable resources (software, server, information, platform, etc.) at any time. The user uses the technology of “thin” client as a means of access to applications, platforms, and data, and the entire infrastructure of the information system is located at the provider of cloud services. Fog computing - is the virtual platform of distributed computing and data storage services on end-terminal devices and network services for data transmission, storage, and processing [15]. Computation is performed terminal devices with limited computing and energy resources - including controllers, industrial equipment, household appliances with microprocessors equipment, sensor network nodes. Mobile computing is human–computer interaction by which a computer is expected to be transported during normal usage, which allows for the transmission of data, voice, and video. Mobile computing involves mobile communication, mobile hardware, and mobile software. Communication issues include ad hoc networks and infrastructure networks as well as communication properties, protocols, data formats and mobile technologies.

The reasons for the development of a converged model of distributed computing are existing problems in the plan of information processes:

  1. (1)

    The information’s opacity, which is associated with the closed data formats of information systems from various developers.

  2. (2)

    Mismatch of sensor data and protocols that are associated with the use of the manufacturers proprietary systems for collecting and data processing.

  3. (3)

    Duplication and synchronization information. Similar sensor data necessary for solving various tasks are duplicated in databases in its own storage formats.

  4. (4)

    Mismatch of sensor data streams, associated with the use of different network technologies and telecommunications solutions.

To solve the problems, the cloud, fog, and mobile computing technologies are used. The convergence networks architecture may include five hardware and software levels:

  1. 1.

    The sensor nodes are associated with industrial controllers and sensors, directly implementing fog computing.

  2. 2.

    Clusters network segments with coordinators, cellular modems, router, which collects and transfers sensor data into the data warehouse.

  3. 3.

    Cloud computing clusters.

  4. 4.

    Warehouse of sensor data and monitoring results.

  5. 5.

    The user mobile devices for the organization of access to computing and information resources.

The convergence results are defined more specifically as the coming together of telecommunications, computing, and broadcasting into a single digital bit-stream.

4 Tools for Monitoring Road Transport Infrastructure

Intelligent monitoring road infrastructure and transport is based on collection and analysis of sensor multimedia data that are collected from various ground platforms, air and space surveillance equipment. As a ground-based platform for collecting information, road video cameras and photo and video fixing tools for traffic violations are used, for example, photoradar complexes with photo-fixing “Cordon” and complexes with video recording “Cordon-Temp”. Complexes can allocate and recognize objects in photos and in a video stream (Fig. 2).

Fig. 2.
figure 2

Operation of photoradar systems

Full size image

The main functions of the complex is a measure vehicle speeds in the control zone, automatically capture and save violators’ photos, recognize license plates and events, collect and transmit such information to a data center, as: recognized number, fixed speed of the vehicle, type of violation, direction of movement, date and time of the violation, value of the permissible speed, name of the controlled area, geographical coordinates, etc. A large number of the data makes it necessary to develop a monitoring system with multi-agent distributed data processing and intelligent analysis tools for forecasting of traffic situations using multi-agent approach [16], the model converged computing, streaming technologies for sensor data processing and machine learning algorithms (Fig. 3).

Fig. 3.
figure 3

Distributed processing and data mining system

Full size image

Monitoring system with multi-agent distributed data processing and intelligent analysis tools performs the following functions:

  • detection and identification of moving vehicles on the controlled road section with speed measurement;

  • automatic photo-video recording of violations of traffic rules;

  • data about traffic flow collection in all controlled areas and transfer to the data processing center via communication channels;

  • vehicles detection and tracking them with visualization of routes of their movements on a cartographic basis;

  • photographs and video materials processing about road accidents and violations;

  • statistical data accumulation and processing about violations for the time periods to identify and analyze changes in dependency violations of traffic accidents from the influence of various factors (weather conditions, traffic volume, repair work, city events, time of day, seasonal factors, etc.);

  • spatial analysis of offenses to identify critical areas and “bottlenecks” in the road transport infrastructure and their dependence on changes in traffic conditions with visualization on the map with color differentiation zones;

  • data mining of offenses for the purpose of forecasting the development of traffic situations and decision-making to improve traffic safety.

5 Multiagent Method of Data Mining from Photoradar Complexes

The multiagent method involves the use of software agents to solve problems of alerting road users about the current and projected traffic situation in the location areas through mobile communication applications and vehicle navigation equipment [17].

A lot of agents work with the expert server component to provide users with alternative routes and recommendations for choosing the best route with customization to the wishes of users. Set of agents includes driver alert agents, warning traffic agents on the routes (collisions, speed modes, traffic lanes, traffic signs, markings, etc.), parking zone information agents, road service information agents, information agents about loading areas for trucks, etc.

Processing sensor data obtained with photoradar complexes can be implemented in the fog computing layer by agents that are loaded into integrated sensor nodes. Wireless sensor nodes ZigBee [18] are connected to complexes for solving simple problems of analyzing photos and video data, search and detection of vehicles in video streams, localization and tracking their movements.

The method of multi-agent processing of sensory data includes:

  • a model of convergent processing of big sensor data,

  • a way to load software agents into sensor nodes,

  • a technique for analyzing time series for forecasting possible situations,

  • a technique of detection, localization, and tracking of vehicles,

  • a technique of remote diagnosis of photoradar complexes and the forecast of performance parameters.

The hypervisor management is consolidated computing resources in a multiprocessor system for distributed processing of big sensor data. Software agents are operated in the sensor nodes and interacted with the data acquisition modules, other agents, and brokers. In this model, computing agent is software template for parallel processing.

The intelligent agent responds to requests, decides on the selection of data processing functions, clone and migrate to other nodes in the network. The agents exchange messages with each other and brokers, which send protected data to the central network coordinator, and receive from them control commands.

The model of intellectual brokers is offered to agent interaction with server applications at the data center. The broker is an agent that runs on routers and realizing the storage, data protection, and transmission functions. The message Query Telemetry Transport protocol is used for the implementation of information exchange with limited energy resources [18]. Collect cloud computing results entering from sensor segments ZigBee network [19] is performed by using the broker MQTT, loaded into computing network gateway cluster, that provides interaction ZigBee protocol stack and MQTT-client. The gateway is implemented on a central coordinator ZigBee network or modem pool cellular network. The broker’s functions are sensor data and aggregates processing, entering in the coordinator; conversion frames with this information to be integrated into the data warehouse; data encryption; support «sliding» window algorithm to ensure reliable transmission, etc.

The results of data processing by agents in the process of monitoring traffic accidents are transferred to the data center, where more complex spatial and intellectual analysis tasks are solved in the second layer of cloud computing and storage.

6 Results of Monitoring

Consider the results of monitoring and analysis of traffic accidents, fixed by an intelligent monitoring system with photoradar complexes, which are installed on the main highways of the Penza region. To accumulate statistics, their spatial and intellectual analysis, synthesis of graphs and reports to support decision making, the system employs a special agent for remote polling of photo-video fixing complexes and automatic upload of data on administrative violations (Fig. 4). This software agent is provided by developers of photo-video complexes “Cordon-Temp”.

Fig. 4.
figure 4

Operation of the software upload agent

Full size image

The agent collects and downloads multimedia data such as photos and frames with violators from the video stream, as well as various telemetric data on traffic parameters, for example, time series with changes in the average speed of vehicles in the controlled area. These time series are represented by the visualization agent in the form of graphs (Fig. 5).

Fig. 5.
figure 5

Time series of average speed of vehicles

Full size image

The results of automatic collection and unloading of photo-video data are recorded in the database during the parsing process (Fig. 6).

Fig. 6.
figure 6

Parsing of paged data by the software agent

Full size image

The database is hierarchical and is located on the data center servers according to the cloud storage model (Fig. 7).

Fig. 7.
figure 7

Fragment of the physical layer structure of the hierarchical database

Full size image

The data mining results and reporting data are also stored in the hierarchical cloud storage database in the server cluster of the data center. To provide operators and other decision-makers with data, the unloading data and monitoring results are extracted from the storage and presented as a data mart.

As an example, consider the results of the intellectual analysis of unloading data collected from 10 complexes in a month’s time in comparison with meteorological data in order to reveal the patterns of variation in the number and severity of road incidents. Figure 8 shows the graphs of the dynamics of the detected incidents by 6 complexes in a month.

Fig. 8.
figure 8

Graphs of the dynamics of road incidents by 6 complexes from 22 March to 22 April 2017

Full size image

Analysis of the data presented on the graphs showed some anomaly. It is seen that for a month on 5 complexes (KD0173, KD0174, KD0183, KD0122, KD0180), the number of road incidents is fixed, which on the average is about 60–70 units with the exception of the KD0201 complex, which is installed. However, after April 17 there is a sharp decrease in the number of road incidents simultaneously at all the complexes. Similar results are visible on the graphs of administrative offenses identified by other complexes. Meteorological data were collected to determine the causes of an abnormal decrease in incidents and offenses.

Comparison of these data and graphs showed a decrease in the level of administrative offenses recorded by Cordon-Temp devices on the M-5 (Ural) route after April 15. These days, unfavorable weather conditions (rain, snow, low temperatures) have led to icy conditions. Such results were achieved as a result of the pilot operation of the prototype of the intelligent analysis system, when after the analysis of the data from the photoradar complexes and meteorological data a forecast of unfavorable situations was made. Next, within a few days, push messages were sent to mobile communications with the results of the forecast to warn drivers about the need to be cautious after April 15. As a result, the road traffic decreased by 20% and the average speed of the controlled sections of the road, where the photoradar complexes were installed, decreased by 40%.

7 Conclusions and Future Work

Currently, there is a prototype of a monitoring and intellectual analysis system with a limited set of functions. The system functionality is implemented by several agents that perform data cleaning, clustering of incidents, comparing time series of road accidents with meteorological data, downloading data from the complexes, retrieving data from the storage for visualization in a data mart in the form of a dynamic hypertable, preparing charts and reports, generating push notifications to mobile client for the prevention of drivers, etc. [20, 21].

In future research, it is planned to develop agents for spatial and intellectual analysis, a road incident prediction agent depending on human mobility and mobile alert agents for drivers for satellite navigators and transponders.

In the process of spatial analysis, similar sections of the road and transport infrastructure are identified by the number and type of traffic accidents. Clustering of such sites allows to define clusters of the most critical and emergency sites and to present them on a cartographic basis with color differentiation of dangerous zones for motorists. The information is transmitted to the server application, which in turn shall forward a coordinate zone information to mobile agents that are downloaded to the user communications. The mobile application-agent interacts with the GPS/GLONASS satellite navigation module and warns drivers about the need to observe extreme caution when entering the critical zone.

In the process of intellectual analysis of time series with the moments of road incidents, time intervals are determined, in which an abnormal deviation of the number of incidents from average indicators occurs. Next, various data are collected on environmental changes at these time intervals. Such data include the results of meteorological observations, information on past events, information on repair work in controlled areas, a calendar of working hours, information about seasonal factors, etc. As a result of the comparison of time series and various factors, factors that are highly likely to become determinants for an abnormal change in the traffic situation in various controlled areas are identified. The goal is to identify critical time zones and factors that cause the occurrence and implementation of road accident risks. The results of the analysis are transferred to decision-makers and mobile users’ applications to warn about the increase in the risks of incidents at specific times and the factors of these risks.

To infer traffic accident, a direct way is to predict whether it will happen or not. However, by performing some analysis on traffic accident data, we have found that it is difficult to forecast the occurrence deterministically under given conditions since traffic accidents are caused by complex factors. Some of these factors such as driver’s maneuver and distraction cannot be observed in advance. Therefore, we have decided to diagnose the risk of traffic accidents. The risk of a traffic accident can be reflected by frequency and severity. We define risk level as the cumulative severity of set traffic accident record. For example, the risk level is 6 if six injury accidents have happened or one fatal accident has happened in the control zone of the photoradar complex in the corresponding time interval. Regions with highest risk level can be regarded as critical zones for the time interval. Traffic accidents are possible to happen with human mobility, like walking, biking or driving, which can be reflected by the density of GPS records. Therefore, another goal of the intellectual analysis is to identify how human mobility will affect traffic accident risk, to develop a model for forecasting traffic accident risk with real-time data.