Keywords

1 Introduction

With massive increases in the world’s population and nearly 70% of the world population projected to live in urban areas by 2050, cities face serious urban planning challenges [1]. Not only do they face rapidly growing population, but they also have to deal with social and sustainability challenges. To better cope with changes, cities need long-term approaches leading to sustainability [2].

Rethinking cities to not only efficiently manage their current situation and population, but also their future growth is exactly the main motivation behind the concept of smart cities. While there is no consensual definition of what a smart city is [3], it commonly involves the usage of Information and Communication Technologies (ICT) to design tools which should respond to people’s needs through sustainable solutions for social and economic challenges.

Bibri and Krogstie [4, 5] proposed an interdisciplinary literature review of smart and sustainable cities and pointed out the interest of a new generation of urban planning tools for improving mobility and accessibility. A smart city is then a significant tool for municipalities which can reduce their spending and perform real-time monitoring of their transportation, energy and utilities networks [6]. These planning tools require a large amount of data that can be nowadays collected via different type of sensors and devices deployed within the city.

Currently, many cities around the world are rapidly developing their existing CCTV network. These large surveillance networks represent a major cost for the councils in terms of maintenance, but are only used for investigating incidents and monitoring anti-social behaviours in public places [7]. This is due to stringent privacy regulations, as only the police and a few accredited operators are allowed to view the live or recorded video feeds. This results in the expensive collection of a vast amount of rich data that have been mostly unused so far.

This paper presents a new framework for real-time traffic simulation integrating an agent-based methodology with existing CCTV data, effectively addressing the issues of the surveillance camera maintenance cost by adding new usages while respecting the privacy regulations.

The paper is organized as follows. Section 2 introduces the challenges of obtaining traffic counts. Section 3 describes the Liverpool Smart Pedestrian project monitoring mobility and air quality within the Australian city of Liverpool. Section 4 then details the visual sensor used to capture the traffic flows within the city. Section 5 describes the use of the collected data within an agent-based traffic simulation. This is followed in Sect. 6 by preliminary results of the approach applied in Liverpool. Concluding remarks and perspective are presented in the last section.

2 Collecting Traffic Counts

The first stage for monitoring and modelling traffic in a road network is collecting traffic counts. Inductive loop detectors, pneumatic road tubes, and temporary manual counts have been the primary methods for collecting such traffic data. The development of automatic sensing technologies, to replace manual counting, has allowed a higher frequency rate as well as the permanent monitoring of the traffic counts [8]. Other classic traffic counter devices include piezo-electric sensors and radar-based off-roads sensors [9, 10]. While initially being designed for vehicular traffic, most of them can also be adapted to count bicycles and pedestrians [11].

With the drastic reduction in the cost of electronic components, and recent advances in machine learning and image processing, it is now possible to develop at relatively low cost, edge computing solutions to monitor traffic. For example, Gupta et al. [12] designed low-cost hardware using Wi-Fi strength as a signal to monitor traffic. The passage of a car between a transmitter and a receiver produces a variation in signal strength that can be measured to count vehicles flows. However, this approach still needs new infrastructures. Another approach is to rely on already existing infrastructures to perform real-time monitoring. Indeed, as cities have been massively investing in CCTV networks [13], retrofitting the already existing CCTV infrastructure to transform classical CCTV into smart CCTV becomes a promising approach to real-time monitoring of traffic. Consequently, more and more research is being done using video analytics on CCTV footage. For example, Kim et al. [14] used CCTV in an urban traffic information system to determine traffic speed and volume, and combine this information with on-board wireless equipment to estimate travel speed.

Finally, when using CCTV footage, ensuring privacy is a major issue. As noted by Satyanarayanan et al. [15, 16] and Shi et al. [17, 18], the edge computing paradigm offers a way to process data at the edge of the network to address concerns such as bandwidth saving, as well as data safety and privacy. Indeed, the privacy of the data is ensured by the processing which denatures the raw data [19]. The resulting transmitted data has typically a significantly smaller size than the original raw data as it contains only the relevant information for the application.

The design of an edge computing device with the ability to perform real-time analysis of CCTV would then allow not only to collect data but also to ensure privacy as the image feeds would not leave the already existing CCTV network and only denatured data would be produced by the device. This is the approach retained in the project briefly introduced in the next section.

3 The Liverpool Smart Pedestrians Project

The Liverpool Smart Pedestrians project was funded under the Australian Government Smart Cities and Suburbs Program for a duration of one year starting from February 2018. It was a collaboration between the Liverpool City Council and the University of Wollongong. The project aimed to design innovative solutions for the collection of data in a non-intrusive way to help inform urban planning in Liverpool, a suburb of Sydney in New South Wales, Australia [19].

The city is growing rapidly, with more housing, offices and educational facilities. The council’s redevelopment of its CBD is expected to bring in 30,000 additional pedestrians per day. All of this makes the city a good area for experiments monitoring the effect of this redevelopment on the traffic.

The results of workshops conducted with the community and the feedback from the city urban planners highlighted the need for sensors monitoring the traffic with the following requirements:

  • Multi-modal detection and tracking: The sensors need to be able to detect and track pedestrians, vehicles and cyclists.

  • Privacy compliant: As sensors are going to be deployed over a city, the sensors should be privacy compliant, meaning that no personal data should be stored or exchanged. Since no raw image will be saved by the device, nor transmitted to a centralised server, there are no privacy issues.

  • Leveraging existing infrastructures: As cities already make huge investments on CCTV systems [13], the solution should take advantage of the already existing infrastructures in terms of networks and cameras. Retrofitting the existing CCTV network to collect more data has been identified as a major innovation.

  • Scalability and interoperability: New sensors can be added at any time, regardless their technologies, meaning the sensor network can be easily expanded and capture new type of data.

Thus, the project aims were to develop and evaluate mobility trackers using CCTV live feeds. Twenty visual sensors have been deployed over the city centre to monitor traffic flows. Fifteen of them use already existing CCTVs while five of them are new mobile CCTV units allowing relocation if needed.

These new visual sensors are capable of tracking and automatically differentiating various types of traffic components such as cars, buses, bicycles or pedestrians in real-time. The processed data is then transmitted to a centralized database and can be visualized on a dashboard in real-time or used to develop an agent-based simulation to infer in real-time the dynamics of the traffic flows in the road network.

As part of this pilot project, 20 air quality and noise sensors are also co-located with the mobility trackers to evaluate the impact of the traffic on air quality and noise pollution. Figure 1 displays a map of the town centre and the location of the sensors.

4 An Edge Computing Device for Traffic Monitoring

The objective of the project was to deploy a fleet of these sensors enabling citywide traffic monitoring in real-time. For that purpose of monitoring the mobility within a network, we have designed a sensor that is able to detect and track entities of interest in a live video feed using intelligent video analytics. The most important feature of the sensor is that it follows the edge computing paradigm, i.e., the video analytics are run directly on the device and only the results of the processing are transmitted. This has two main advantages:

  • It lowers the network bandwidth requirement as no raw images are transmitted, but only indicators and meta-data; and

  • Thanks to the limited amount of information being transmitted, the device is privacy compliant.

The privacy compliance of the device is critical for real world applications and deployment in smart cities. Indeed, the system can be paired with existing CCTV infrastructure while not transmitting the actual video feed captured from the cameras. This lowers the deployment cost of the sensor as no additional camera is needed while allowing uses of the already existing CCTV infrastructure.

Fig. 1.
figure 1

Location of the visual sensors (triangles), and the air quality sensors (circles) in Liverpool.

Full size image

The sensor performs the following steps iteratively on average 20 times each second:

  1. 1.

    Frame acquisition;

  2. 2.

    Detecting the entities of interests in the current frame via YOLOv3 [20];

  3. 3.

    Tracking by matching the current detections with the ones in the previous frame via the SORT algorithm [21];

  4. 4.

    Updating the trajectories of entities already stored in the device database or creating records for the newly detected objects.

In parallel, the device regularly transmits the outputs, i.e. the trajectories and number of entities tracked, either over Ethernet or LoRaWAN networks, the latter being a wireless long range, low power network well suited for the Internet of Things. The interested reader can find a complete description of the visual sensor in [19].

5 Using Live Data in Traffic Simulation

The proposed model receives in real-time data from the sensors which is used to generate origin-destination matrices to estimate the demand on the network. The origins and destinations correspond to the sensors’ locations in the network which will be thus acting as generator of travelling agents and attractor for those agents. A newly generated agent will then compute a path to a randomly selected destination. The random draw is weighted against the traffic count observed by the sensors. It should be noted that a travelling agent can represent either a pedestrian, a bicycle or a vehicle as the smart visual sensor is able to detect and differentiate those three types of entities.

The class diagram of the model, Fig. 2, shows the interaction between the different agents. Note that the air quality sensors and CCTV sensors are not merged into a single class, as they do not share the same attributes. Like Rodrique et al. [22] we make the difference between fixed agents, in the sense that they do not affect the movements during the simulation, and travelling agents:

  • Fixed agents: 

  • CCTV Sensor: They represent the smart visual sensors in Liverpool. The travelling agents are created and removed at the sensors’ locations.

  • Air Quality Sensor: They represent the air quality sensors in Liverpool. The travelling agents are assigned pollution exposure based on those sensors.

  • Traffic Signal: They represent the traffic lights in the road network. The travelling agents stop at the traffic signals depending on their colour. If the frequency of traffic light changes is known, the simulation is able to predict the roads chosen by the travelling agents (pedestrians/vehicles).

  • Road: The roads of the road network on which the travelling agents move. Note that in the model, a difference between the roads for pedestrians and the roads for vehicles is made, as some roads are only accessible for pedestrians and not for vehicles. In our class diagram, we merge both roads (roads for vehicles and roads for pedestrians) into one agent road, as they share the same attributes, e.g. direction, name, length, etc. 

  • Travelling agents:  These agents travel between CCTV sensors and include: 

  • Vehicle: These agents move along the roads at a certain speed.

  • Pedestrian: These agents represent the pedestrians walking within the network.

  • Bicycle: These agents illustrate the bicycles.

For simplification purposes, the three different moving entities are merged into one agent TravellingAgent in the class diagram (Fig. 2). Finally, the conceptual model drawn for the multi-agents simulation is illustrated in Fig. 3. Figure 4 depicts the creation of the travelling agents using the following steps:

  • When the time in the simulation \(t_{simu}\) equals to the time of our data \(t_{start}\), our simulation compares the number of travelling agents (pedestrian/vehicle) seen in the simulation (\(\# TS)\) to the number of travelling agents seen by our smart visual sensors (\(\# TR)\). If more travelling agents are seen in our data than in our simulation, i.e. \(\# TR> \# TS\), then create the number of missing agents in our simulation. If, on the other hand, our simulation sees more agents than our sensors, i.e. \(\# TS> \# TR\), then remove that exceeding number of agents from the simulation (as it would mean that less travelling agents were recorded by the sensors). For each travelling agent created at the different sensors, a destination (corresponding to a sensor) is randomly created.

  • As we increment a line in our data, we also increment the time, and the travelling agents move on their path (the distance of their movement d will depend on their speed v using \(d = v \times t \)).

  • If the travelling agent’s location in the simulation (\(x_{TS_n}\), \(y_{TS_n}\)) corresponds to its destination (\(x_{D}, y_{D}\)), i.e. the agent has arrived to its destination, then remove the travelling agent from the simulation.

  • End the simulation if all the data has been read.

The implementation relies on the GAMA platformFootnote 1, a free and open agent-based framework with a strong focus on spatial simulations.

Fig. 2.
figure 2

Model class diagram

Full size image

6 Preliminary Results

Using the data collected from the smart sensorsFootnote 2 and the zone data obtained through OpenStreetMapFootnote 3, we construct a model simulating the traffic flow in the city of Liverpool in real-time. This model allows us to infer the interactions between the different type travelling agents (pedestrians, cyclists, vehicles) and the fixed agents (sensors), as illustrated in Fig. 5.

Fig. 3.
figure 3

Design model of the proposed traffic simulation.

Full size image
Fig. 4.
figure 4

Activity flowchart. Used notations: \(t_{simu}\) is the time in the simulation, whereas \(t_{start}\) and \(t_{data}\) are the times in the data. \((x_{TS_n}, y_{TS_n})\) represent the location of the travelling agent at \(t_{simu}\), and \((x_{D}, y_{D}) \) the coordinates of its destination.

Full size image
Fig. 5.
figure 5

Traffic network: the red and green circles depict the traffic lights, whereas the blue circles represent the visual sensors. The blue squares represent the vehicles, the yellow triangles illustrate the pedestrians. The date and time of the simulation captured in this picture is 9/11/2019 at 3.02PM.

Full size image
Fig. 6.
figure 6

Traffic flow: number of pedestrians detected by the visual sensors (green) vs number of pedestrians seen by the sensors in the simulation (red) per second (Color figure online).

Full size image
Fig. 7.
figure 7

Exposure of the travelling agents to particulate matter (PM) 2.5 on 10/11/2019. The redder (or greener) the color of the cell, the higher (or lower) the PM 2.5 rate is (Color figure online).

Full size image

Furthermore, behaviours such as congested roads at certain peak times, points of interests of the road network, exposure of the travelling agents to pollutants, and the evolution of the number of travelling agents in the traffic flow, can be inferred. This is illustrated in Fig. 6.

As it is driven by the data coming from sensors, the proposed model is able to simulate the traffic flows of Liverpool at the macro-level (i.e. at the level of the sensors) and can thus be used to monitor and predict different behaviours of the traffic flows, while respecting the privacy of the citizens.

As mentioned earlier, air quality sensors are also deployed within Liverpool. It thus possible to use their data and Air Quality agents to determine the air pollution exposure of the travelling agents. This is illustrated in Fig. 7.

7 Conclusion and Future Work

Being able to accurately estimate and forecast traffic flows across urban networks has become crucial for transport managers and urban planners to create more liveable and caring cities [23]. Currently, traffic monitoring mostly still relies upon inductive loop detectors associated with SCATS, an adaptive urban traffic control system adopted by many Australian cities [24]. These detectors are sparsely located at key crossroads in order to inform the syncing of traffic lights across the city. However, their use for proper traffic estimation and simulation is limited and often must be complemented by other means, including manual surveys.

In this work, we are using a new sensor performing real-time traffic monitoring using existing CCTVs, while respecting the privacy regulations. The live data collected from the sensors is then used in an agent-based simulation model to infer the traffic dynamics in real-time. This method can be easily extended to include other types of sensors and data, such as air quality and noise pollution.

The proposed approach does not only deliver a novel approach to accurately monitor and predict different type traffic flows, their interactions and the resulting pollution in cities, but has the potential to be applicable to many other situations where the structure of the network and some partial and local information is known, but the global dynamics are still unknown.

Further work includes taking into consideration the weather data in order to refine the model of the distribution of the air pollutants measured by the air quality sensors [22], establishing a live connection to the data, using more realistic traffic light frequencies, as well as improving the current traffic model by using strategic agents [23] and a state-of-the-art car following model [25].