Keywords

1 Introduction

The rapid and ever-increasing population and urban activities has imposed a massive demand to urban transportation systems. The main problem is that most of the urban areas were not prepared for such hasty development which led to weak and defective metropolitan transportation systems [9]. Efficient transportation systems are crucial to an industrialized society being its main communication infrastructure; therefore rapid and effective interventions in traffic management and planning are needed to prevent their negative impact on the city’s social and economic welfare. Therefore, by using simulation and taking advantage of its characteristics we can test several possible solutions or even changes in the network more cost-effectively and faster. Indeed, simulation approaches can provide us with the possibility of comparing studies between new infrastructures designs or control algorithms without having to interfere in the real world.

Also, one important characteristic to bear in mind is that the domain of mobility (transportation of both vehicles and persons) presents an inherent complexity. It involves diverse heterogeneous entities either in structure or in behaviour, e.g. vehicles, pedestrians, traffic system, among others, which can interact, reflecting social behaviours that goes from coordination and collaboration to competition. Moreover, a high degree of uncertainty and dynamism especially when considering the urban context is uncovered.

To address the rising issues of these new trends a new generation of mobility systems emerged with the advent of what has been coined Intelligent Transportation Systems (ITSs), forcing architectures to become adaptable and accessible by different means so as to meet different requirements and a wide range of purposes. ITS arises as the synergy between the information and communication technologies (ICT) and the urban transportation systems, which include vehicles and networks that transport people and goods. The idea of such systems is to ensure the efficient utilisation of the available road capacity by controlling traffic operations and influencing drivers behaviour by providing proper information and stimuli.

The formalization of the ITS concept is to be considered a great achievement by the transportation engineering, practitioners and scientific communities. The explosion of the computing technology in terms of applications experimented in the last couple of decades brought together expertise from different scientific and technical disciplines giving birth to new computing and communication paradigms. A new type of systems coined as socio-technical arose from such mutual conjunctions where people and technology live in mutual symbiosis. The transportation and, generally speaking urban domain, could not be impermeable to such revolution. Indeed, it proves to be a valid field where new social and technological paradigms emerge. A new concept has been concocted to deal with this revolution, the so called future urban transportation (FUT) systems. The notion of mobility systems within FUT overcomes ITS limitations; instead of focusing only on the simple processes of transporting goods and persons they become self-conscious in terms of environment, accessibility, equality, security, and sustainability of resources [16]. People are placed as a central aspect, as well as are their preferences, of the urban systems, forcing architectures to become rather adaptable and accessible to their needs. Therefore, new technologies and methodologies are necessary to support these new models, which motivates this work.

Normally in the development of traffic solutions, the use of a simulator is very straightforward related to traffic flow and junction management. In spite of many attempts and published papers, the solutions presented do not make full use of the concept of intelligent agents. Additionally, the multi-agent systems (MAS) metaphor has become recognized as a useful approach for modelling and simulating complex systems [14].

These new perspectives in urban mobility systems disclosed the need for the design of more human-centric and sustainable solutions. A framework that is capable of generating urban contexts (meaning a traffic network, infrastructures and the population of commuters) is definitely necessary so that analysts and designers can study, develop and evaluate their policies and strategies.

In this paper, we present a framework that meets all these requirements, providing practitioners and scientific communities with a tool that can instantiate an artificial society (AS) of heterogeneous drivers and intelligent traffic light management solutions, immersed in a realistic traffic environment. The concept of AS can be used by traffic managers or government institutions as a test-bed for the analysis of strategies and policies towards a social-aware and sustainable use of resources. Combining a powerful and standardized MAS development framework, JADE, with a large-scale microscopic traffic simulator, SUMO, allows different types of studies, namely intelligent traffic control algorithms, service design, additionally to studies for the evaluation of new policies and vehicle-to-vehicle (V2V) communication applications.

The remainder of the work is organized as follows. Section 2 motivates this research project and presents some related work, whereas Sect. 3 discusses on the tool-chain used to implement our proposed approach, detailed in Sect. 4. We illustrate our approach in Sect. 5 and draw conclusions in Sect. 6, identifying potential future work and further developments.

2 Related Work

Due to the high complexity and uncertainty of contemporary transportation systems, traditional traffic simulation fails to capture in detail all the dynamics that characterize them. For example, travelers can choose whether to travel or not, can change their planned itinerary at any moment, and their choices may be affected by any social, economic or environmental phenomena. Also, new performance measures brought about by an extensive future urban transport agenda and the implementation of the concept of smart cities pose additional requirements to which the user is central, not as easily integrated in traditional modelling approaches.

In order to appropriately represent, test, and analyse transportation control and management strategies, Fei-Yue Wang devised and introduced the concept of Artificial Transportation Systems (ATSs) [12, 24]. Basically, ATS goes beyond traditional simulation methodologies and integrates the transportation system with other socio-economic urban systems with real-time information resulting in a powerful tool for transportation analysis, evaluation, decision-making and training. The foundations of ATS are to be searched on the paradigms of multi-agent systems, social simulation and artificial societies, as well as distributed computing, which provide adequate tools to represent interacting entities of complex domains such as intelligent transportation systems. Rossetti et al. [18] provide a brief overview of contribution in ATS development along three dimensions: modelling issues and metaphors for ATS models, architectures for ATS, and practical applications of ATS. However, it results that very little has been advanced in what concerns the appropriate representation of users and their behaviour, in the various dimensions of Intelligent Transportation Systems.

Passos et al. [16] have carried out an evaluation of current available simulation environments and their ability to capture the aforementioned requirements. Their analysis features those characteristics of the future transportation systems where not only performance is essential but also the user entity is regarded as a key aspect playing an imperative whole in all social interactions taking place in such a complex domain. Among the desirable features that both works suggest is the agent-orientation of the candidate platform.

Although major traffic simulation packages and tools implement various important and advanced features, they still treat vehicles and drivers indistinctly following traditional modelling approaches such as car-following, lane changing and adopting a normative rather than a truly cognitive behavioural approach, reflecting users’ decision-making and their preferences.

In the literature, some similar approaches can also be found that apply the agent metaphor to traffic simulation. ITSUMO [21] implements a cellular-automata approach and is formed up by four distinct modules, namely the data module, the simulation kernel, the driver definition module, and the visualization module. The agent metaphor is used in the sense it is possible to define driver decision-making procedures that simulate human-like cognition processes. The simulator also offer apropriate tools to test with intelligent traffic control strategies.

Balmer et al. [1] present the MATSim framework as a suitable tools for large-scale agent-based transportation simulations. In MATSim, each traveler of the real system is modeled as an individual agent and the simulator integrates activity-based demand generation with dynamic traffic assignment. The traffic dynamics is simulated using a macroscopic resolution of the transportation domain, whereas an activity-based demand approach models daily activities as diaries of trips for every “agent” in the population; each agent then performs journeys according to her own activity diary resulting in the network dynamics.

Within the Agentpolis project [10] it has been suggested a modular framework for the implementation, execution and analysis of simulation models of interaction-rich transport systems. The framework fully adopts the agent-based modelling paradigm, which makes it very versatile and capable of modelling systems with complex ad-hoc interactions and just-in-time decision-making.

Rossetti et al. [19], discusses an integrated multi-agent system that applies a methodological approach that allows for the assessment of today’s intelligent transportation solutions through the metaphor of agents through a truly agent-directed simulation perspective. Their work conceptualizes the application domain in terms of agents and three basic subsystems are identified, namely the real world, the virtual domain, and the control strategies inductor that actually conducts the simulation process.

We believe however that it is possible to separate the drivers’ decision-making and the vehicle control obtaining a clear separation of the supply (network) and demand (drivers choice) layers on the basis of the so-called delegated-agent idea. Following the MAS paradigm, drivers’ decision-making process and choices will be embedded into a driver agent while the simulation platform implements the environment as well as the traffic infrastructure. This approach will allow planners to make a better design of the concepts envisaged by the new generation of urban transportation systems also under the perspective of the encompassed socio-technical aspects.

Similarly to our approach, ATSim [7] is presented as a multi-agent-based traffic simulation system to support global system throughput on a macro-level view, whereas individual vehicle decision-making is kept decentralized and separate from the traffic flow simulation itself. Thus, the infrastructure elements of the traffic domain, such as traffic lights and vehicles can make use of the agent paradigm with a reasonable performance. The system consists in coupling the commercial traffic simulation suite AimSun with the JADE platform for the development of multi-agent systems. In our approach however, we have opted to perform the traffic flow simulation using the open-source SUMO, instead of using a commercial simulator.

3 Tools

We propose a framework that allows us to build an artificial transportation system that represents all the entities composing it: a population of drivers feature deliberation abilities and situated in a road traffic environment. We face the problem of coupling two resolutions of the traffic system: one nanoscopic that reflects the decision-making module of a driver, and another microscopic traffic model reflecting vehicle interactions. It is obvious that we need to combine and synchronize different tools to achieve such multi-resolution setting.

The traffic simulation tool needs to implement the necessary concepts of the transportation domain (or to provide flexibility for additional implementation of them) and to provide a proper interface for controlling and monitoring the simulation entities and states. We will also need an intuitive MAS development framework, which will be used to implement the artificial society of drivers and various environment artifacts, such as the advanced traveler information systems (ATISs) and the intelligent control infrastructure.

In order to implement our requirements we opted for using the SUMO traffic simulator to represent the road network with the vehicle and the traffic control (physical) infrastructure. The JADE platform is used to represent the multi-agent systems composed of drivers (and generally speaking a synthetic population of travelers) and intelligent traffic management services (e.g. ATISs, intelligent traffic lights, and so forth). Finally, the TraSMAPI application is used to allow the synchronization between the agent-based population and SUMO.

3.1 SUMO/TraCI

A very popular tool to the traffic and transportation research community is the SUMO (Simulation of Urban MObility) traffic simulator. SUMO is a suite of applications that are used to design and implement realistic traffic simulations [11]. It represents both the road network infrastructure and the traffic demand and it has been used in several research problems such as route choice [8], traffic light algorithms [13], simulating vehicular communication [17], among others. The popularity of the simulation suite derives from the fact that it is open-source, highly portable, and offers microscopic and multi-modal traffic simulation packages designed to handle large road networks and to establish a common test-bed for implementing algorithms and models for traffic research. It is also stable and in continuous evolution supported by a large community of developers and users. Besides the aforementioned mentioned features it also facilitates interoperability with external applications in run time using TraCI (Traffic Control Interface), which allows developers to access a running road traffic simulation. TraCI uses a TCP-based client-server architecture providing access to SUMO. It opens a port in a SUMO simulation and waits for outbound well-defined commands, offering us with a wide range of features to use while the simulation is performed.

Nevertheless it still experience drawbacks from the traditional approach of dealing with vehicles and drivers indistinguishably. Although it can be extended due to its open-source nature, this obliges the user to directly implement patches to add any new functionality.

3.2 JADE

JADE (Java Agent DEvelopment Framework) [3] is a free software framework to develop agent-based applications. Its goal is to simplify the development of MAS while ensuring standard compliance through a comprehensive set of system services and agents. JADE is fully implemented in Java and is compliant with the Foundation for Intelligent Physical Agents (FIPA) specifications for interoperable multi-agent systems. Besides, this agent platform can be distributed across several machines, which do not even need to share the same Operating System (OS).

This framework can be considered a middleware that implements an agent platform and a development framework. It deals with all those aspects that are not peculiar to the agent internals and that are independent from the application domain, such as message transport, encoding and parsing, or the agent life-cycle. JADE’s aim is to simplify the development of multi-agent systems while guaranteeing standard compliance with the FIPA specifications: naming service and yellow-page service comprising the Directory Facilitator (DF), message transport and parsing service, and a library of FIPA interaction protocols ready to be used. All agent communication is performed through message passing, where FIPA ACL is the language to represent messages. The agent platform can be dispersed on several computers, where each of which runs a single Java Virtual Machine (JVM). Each JVM is a container of agents that provides a complete run-time environment for agent execution and allows several agents to concurrently execute on the same host. Each agent is implemented as a single thread, however, agents often need to execute parallel tasks. As JAVA language offers multi-threading solutions, JADE also inherits these characteristics. Moreover it supports scheduling of cooperative behaviours, storing these tasks in a light and effective way. The runtime includes also some ready to use behaviours for the most common tasks in agent programming, such as FIPA interaction protocols [4]. Numerous R&D projects, where an interaction between several elements is required, and in which an autonomous and dynamic adaptation to complex relations is needed, have used JADE as a developing tool. In the traffic domain, there are several works that profit from the JADE platform for developing MAS-based traffic management solutions [20, 22].

3.3 TraSMAPI

TraSMAPI (Traffic Simulation Manager Application Programming Interface) is a synergy between two main components: an Application Programming Interface (API) and a Multi-Agent System framework. The API was built upon an abstraction level higher than most common microscopic traffic simulators so that, ideally, the solution should be independent from the microscopic simulator coice. This is guaranteed as far as the chosen simulators allow it, and provided that their communication interface differs and they do not implement the same set of functions. This feature allows the comparison of results from different simulators using exactly the same implemented traffic management solution.

The multi-agent system framework is a module that is meant to serve as a starting point for the creation of multi-agent systems. It allows the creation of new agents by following a common interface. The agents are created with a reference to one or multiple objects in the simulation gaining direct access to its artifacts or entities. As far as our work is concerned, we aim to replace that MAS framework module with more widely distributed MAS frameworks, such as JADE. This approach allows us to use TraSMAPI also in the implementation of real-world solutions since it will have a more mature, generic and FIPA-compliant MAS development framework.

4 Research Method

We propose a layered approach to represent drivers’ decision-making capabilities within the framework, which are: a strategic layer that encompasses cognitive functions and decision-making processes, and a tactical layer where the basic control of the vehicle resides. In fact, with the previous division, we were able to decouple driver’s cognition from her behaviour (seen as demand) from the traffic simulation that represents only the physical infrastructures (seen as supply).

We want to have agent-based ITS solutions implemented in JADE and operated in the SUMO environment. The path to accomplish so is to have an heterogeneous artificial society of drivers in the JADE agent platform whilst each of these agents is responsible for one vehicle in the SUMO’s traffic environment.

4.1 Integration

Taking into consideration all the general requirements and goals we have devised the following architecture as depicted in Fig. 1. We can observe the main contribution of TraSMAPI in our framework.

Fig. 1.
figure 1

Framework architecture

Full size image

TraSMAPI provides an abstraction over different possible microscopic simulators, which makes our platform completely independent from the simulator used. Besides, it makes possible further studies on simulation results comparisons, since it is possible to test the same solution, i.e. source code, in various simulators, hence demonstrating TraSMAPI’s transparency and self-reliance. In addition to the TraSMAPI block, we can also observe that JADE is directly connected to the microscopic traffic API (TraSMAPI), which has a communication model for the SUMO Simulator that reflects the basic API for the interaction with the simulator.

The microscopic traffic simulator offers an API for access to its simulation state - TraCI. For an external application to communicate with this software it must obey the TraCI communication protocol and messages types. The SUMO Communication Module attached to TraSMAPI converts this low-level simulator API to a higher-level programmer’s perspective, which will be then used by our artificial society of drivers implemented in JADEs MAS development framework coupled to TraSMAPI.

4.2 Driver Agent Architecture

To build and associate each driver agent with a simulated vehicle and endorse her with all driving decisions, skills and cognitive characteristics would be computationally very expensive. To simulate hundreds or thousands of vehicles and drivers’ decision-making in JADE we have adopted the delegate-agent concept, which has been used in [23], to separate the tactical from the strategic layer of the agent, and execute them in parallel, thus improving performance (see Fig. 2).

Fig. 2.
figure 2

Driver’s Layers: tactic-reactive in SUMO; strategic-cognitive layer in JADE.

Full size image
Fig. 3.
figure 3

Vehicle abstraction through the architectural design.

Full size image

The idea of separating the tactical from the strategic layer of a driver agent is based on the different time-scale and complexity of the cognitive and reactive actions related to the primary task of driving. That is, the driver will need a short time to take an action reacting to a traffic event such as accelerating, slowing down, changing lane, or over taking. On the other hand the task of collecting and processing information related to traffic messages or other recommendation necessitates longer time periods. The tactic-reactive layer, following a rule-based behaviour, was entrusted to the microscopic traffic simulator, taking care of reactive actions associated with driving itself. Thus, drivers endow the feature expressed by SUMO’s driver behaviour model, as follows:

  • accelerating and breaking actions

  • lane-changing behaviour

  • anticipating events

The strategic layer, expressed as the route-choice or adaptive learning behaviour, was kept in JADE framework. Here, the researcher can implement her own strategic architecture, from pure reactive to pure cognitive agent architectures. Following are possible high-level reasoning tasks the agent can perform:

  • information assimilation

  • mental map and representation of the network

  • reaction to traffic messages

In order to achieve these ideas, we need to extend the scope of TrasMAPI, enabling it to build an abstraction over the vehicle entity, as illustrated in Fig. 3. We have implemented the communication protocol regarding the methods of a vehicle for variable retrieval or state change taking into account the compliance with the well-defined instructions of TraCI, for further information see TraCI’s documentation and reference [25], where the protocol and message flow are presented and detailed.

4.3 ATIS Artifacts Implementation

The concept of the artifact according to activity theory is to enable action and mediate interactions of the active components in an environment. So, artifacts mediate the interaction among agents, as well as between agents and their environment. On the other hand, artifacts embody the part of the environment that can be designed and controlled to support participants activities [15]. Since artifacts affect the space of agent interaction coordination, cooperation and competition issues can arise. One can easily see how artifacts possibly play a key role in engineering self-organizing systems, as they can be noticed in the traffic domain (such as coordination among drivers for platooning formations, or between adjacent traffic lights to allow green-wave coordination).

By using the agent concept as a programming paradigm to build and model ATIS artifacts we are able to blend them into JADE. The purpose of ATIS is to acquire, process and present information to travelers assisting them in their travel activities. Thus ATIS artifacts can operate as amplifier of the observable space of the driver agent by “extending” its spatial cognition. The agent processes the information and eventually changes her plans according to the input reducing this way travel time. ATIS also facilitates the communication among driver agents as it can receive notifications of events in the network (SUMO environment) and broadcast it back to the whole network. Thus, we can consider this infrastructure as a receiver and a service provider, as we observe in Fig. 4.

Fig. 4.
figure 4

ATIS interaction with environment and drivers

Full size image

The sources, from which the ATIS infrastructures gather information are quite vast. They vary from network data, simulation information and accident notification by drivers, building a common knowledge base on traffic information that can be abstracted as a blackboard. An ATIS agent is responsible for a certain type of work and field of action. We have defined three types of ATIS Agents, the Radio Broadcast (RB), the Variable Message Sign (VMS), and the Informative Traffic Lights (ITL), as seen in Fig. 5. Each ATIS entity uses the microscopic traffic simulator SUMO as a sensory environment. To represent this infrastructure inside the simulator we used the SUMO’s Polygon4 abstraction, which does not have an active role in the simulation though. To gather this sensory data APIs calls must be invoked. However, the mere fact of asking a variable and getting its value can be quite time consuming, since it demands the exchange of two messages, i.e. the request and reading the variable value. Considering this performance issue, SUMO’s TraCI provides two subscription commands that showed to be very useful to retrieve information on network data: variable subscription and context subscription. With these commands, one can register a request for value retrieval for a defined amount of time, which eliminates the request phase, thus reducing the execution time in about half of the original time.

Fig. 5.
figure 5

All three implemented ATIS entities

Full size image

Variable subscriptions provide a periodical update on a structure variable. Context subscriptions allow to obtain specific values from surrounding objects of a certain object within a defined range. This is the reason why we have represented the ATIS entity as a polygon object in the simulated environment. With this implementation we were able to create an entity that gathers facts from the network and simulation data, and provides them to the driver agents through JADEs messaging network.

5 Scenario Illustration

To illustrate the capabilities of the platform we will consider two scenarios where drivers need to make choices over possible routes to follow. The first example shows how agent can decide on-line the fastest route to choose between an origin and destination. The second example is related to the Braess Paradox. Here a synthetic population of drivers adapt by learning their daily routes following a day-to-day (individual) traffic flow analysis.

5.1 Route Choice Example

As a first scenario we have used and improved the network model of the German city of Eichstätt, using the JOSM application to correct some intersections and lane cardinality (see Fig. 6). Graphically we added the Google Maps decals to improve the user immersion during simulation visualization. The configuration files used to load SUMO simulation are only the network file and GUI settings. In this experimental set-up we intend to reproduce the drivers’ decision-making process in route choice based on previous travel times.

Fig. 6.
figure 6

Eichstätt transportation network

Full size image

With our framework we can instantiate a Driver to each vehicle simulated in SUMO. Therefore one may use all the methods which this simulated instance has, such as change destination, speed, route, among others. In the beginning of the simulation it is given to each entity of the AS a random origin edge and orientation and a different random destination. With this, the agents’ reactive layer in SUMO can make use of its shortest path algorithms and calculate the best route based on travel time to accomplish each driver’s desire.

As a proof of concept, each Driver sets a value to its travel time table in SUMO, so that when the reroute-by-travel-time algorithm is called, it will take into account the updated values in the table. This approach argues in favour of the drivers’ awareness and decision-making capabilities. The instantiated traffic-light entities are an extension of a previous experience concerning advisory-based traffic control [13].

5.2 Braess’ Paradox Illustration

There are generally two types of travel behaviour: user-optimizing behaviour, in which travelers select their optimal route, and are generally characterized as “selfish”; and system-optimizing behaviour, in which a central controller directs traffic. Our work focuses on the former and thus the Braess’ paradox occurs only for user-optimizing behaviours.

In an urban area with a lot of traffic, adding a new road to distribute and facilitate traffic may seem an intuitive idea. However, according to the Braess’ paradox, just the opposite occurs: a new route added in a transportation network actually increases the travel time of all individual travelers [5, 6]. The Braess’ Paradox is a good illustration of how easily our intuition about collective interaction can be fooled.

Car drivers seek to minimise the time to get from origin O to destination D. However, car drivers may not be able to act independently of each other: collective interactions may influence individual behaviour. We have made this as a proof-of-concept experimentation scenario of one of the numerous uses that this platform provides to the community of researchers and practitioners. In this case we tried to replicate the Braess’ paradox by setting up an artificial society of “selfish” learning drivers, in a well-defined scenario. Their goal is to get from point A to point B the fastest way possible.

Fig. 7.
figure 7

Two route network

Full size image
Fig. 8.
figure 8

Three route network

Full size image

The network, sketched in Fig. 7, is an abstraction of a network being composed by two symmetrical routes, each of which consists of a fast section and a slow one. Then at a certain time, a new fastest road is added (Fig. 8) providing drivers more and better road resources. We have built an artificial society of Q-learning drivers, which will “live” for 500 days and perform, each day, a trip from point O to D. When arrived at destination, each driver registers her travel time (TT):

$$\begin{aligned} TT = arrivalTime - departureTime \end{aligned}$$
(1)

Taking the environment into account we have modelled it in a finite-state automaton, with 3 edges from node ‘O’ to node ‘D’, and we have built the correspondent Q-table to each of the driver agents, where each route choice in state s generates a utility.

Since our problem is scalar, depending only on the route choice and not also on the current state, we can simplify it to Q(r), being r the route chosen. Hence our utility-function is:

$$\begin{aligned} Q \left( r\right) = \left( 1 - \eta \right) Q \left( r\right) + \eta .R \end{aligned}$$
(2)

being \(\eta \) the learning rate and R the Reward function:

$$\begin{aligned} R=\frac{aTT}{TT}-1 \end{aligned}$$
(3)

whilst TT is current travel time and aTT is the average travel time of all trips:

$$\begin{aligned} aTT=\frac{\sum TT}{\sharp trips} \end{aligned}$$
(4)

For our test-bed we have defined an exploration and an exploitation time in each network configuration for all 500 days. Each network configuration, meaning different route arrangement, is explored by the driver agents during 50 days, in which the drivers are randomly assigned a route so as to retrieve knowledge from its journey time, thus updating her Q-table. The remaining days are exploited by the driver according to their utility values. Drivers’ departure time is equally distributed along the first hour of the day. So in the two-route scenario the drivers will perform 50 days of exploration and 150 days of exploitation. Afterwards, they will have another 50 days of exploration and 250 days for exploiting their best options.

We have performed several tests with various numbers of drivers to observe their learning process in a route-choice setting and we identified two different patterns. Following, two different setups of traffic density have been considered. First we run the simulation with a very low vehicle density that did not put in evidence the paradox scenario. Because the departure times of the drivers were very temporally sparse the new route has not been jammed and therefore chosen by the majority of the drivers. On the second setup we have explored a scenario with a high density of vehicles, approximately one vehicle each two seconds. In this case, we have noticed the increase in travel times and the underutilization of the additional route, regardless of being the fastest alternative. This experiment is plotted in Fig. 9.

Fig. 9.
figure 9

Occupation and travel time in the 1900 vehicle test. Top: Occupation(trip); Bottom: Travel-Time(trip)

Full size image

In the first exploitation phase [50,200], the number of vehicles that chose route A or route B is nearly the same, without fluctuations, which establishes a constant average travel time (observed in the bottom graph). During the second exploration phase [200,250], we verify that the average travel time in the new route C is a bit shorter. Hence, in the beginning of the second exploitation period [250,500] the drivers should have a great utility in the choice of C. In fact, we can observe that almost every 1900 vehicles chose to travel through it i.e. route C, overpassing the initial average travel time, recorded when there were only 2 routes available. With this insertion, the average travel time increased from approximately 1000 s to a staggering 3000 s.

The learning drivers, encountering such a scenario, quickly changed their option based on the utility of route C. They returned to their previous choice avoiding the overpopulated route improving their travel time. We can observe this event in the quick variation of peaks in the upper plot in just approximately 20 days. With this learning process the overall travel time drops as well as the underutilization of route C, which becomes the less used route, regardless being the fastest one.

However, the purpose of the paper is not to discuss the Braess’ Paradox, as it has already been done by Bazzan and Klügl [2] but to illustrate how the tool can be used in a realistic case study, using a microscopic traffic simulator and an artificial society of adaptive drivers. The adaptation by learning aims to represent knowledge acquisition and exploitation of a network.

6 Conclusion and Future Work

Traffic systems have been subjected to a lot of improvement in the last decades and travelers have, in general, witnessed a revolution in the way a trip is planned in urban networks. Hence, facing the current traffic situation in most developed countries it is now imperative to foster new transportation methods using state-of-the-art technologies towards Future Urban Transport (FUT).

Simulation has proved to be an effective approach to analysing and designing novel traffic solutions in socio-technical systems. We have devised a conceptual architecture of an artificial transportation system based on a well-established platform for the development of multi-agent systems and a popular open-source microscopic traffic simulator. Following the concept of delegated agents we define and implement a two-layered architecture representing an driver agent where we differentiate between the reactive and cognitive capabilities of the agent. To illustrate the potentiality of our approach in representing human behaviour, we built a synthetic population of adaptive drivers, where we experimented the knowledge representation of the network using Reinforcement Learning techniques. Finally, we have shown how the proposed framework can be used to instantiate MAS of different nature over the traffic domain complying fully with the notion of the socio-technical systems and other embedded intelligent artifacts (e.g. ATIS, intelligent traffic control, and so forth).

With respect to microscopic traffic simulators and specifically to SUMO, we extend their capabilities as we can allow the design of truly AI-based solution to be tested in the traffic domain without the necessity of modifying the core of the simulator. Also we have extended the type of possible analysis one can perform. The notion of the 2-layered architecture allows the simulator to “implement” memory and thus we can improve the within-to-day and day-to-day traffic flow analysis considering cognitive and behavioural aspects of the driver based on her own preferences.

Generally speaking the proposed tool also reveals great flexibility for multi-agent systems design and development in the traffic and transportation domains. Developers can easily model and test their own artificial society of drivers, where each agents is presented with her own preferences and beliefs. Such artificial society can thus be used to design solutions based on individual or collective intelligence and participation (social-awareness) or as a test-bed for policy evaluation by governmental institutions and decision-makers. This approach will help practitioners to design and test more human-centric, cost-efficient and environmental sustainable solutions. As future developments, not only vehicle-to-X (V2X) scenarios but also the development of new policies and incentive mechanisms studies might be carried out and evaluated through our platform.