Keywords

1 Introduction

This paper describes the work performed for the evaluation work package of H2020 ROBORDER Project and reports it in accordance to the standardised practice described in the DSEEP (Distributed Simulation Engineering and Execution Process) process [1], steps 3 to 7 (Fig. 1).

Fig. 1.
figure 1

DSEEP and VV&A Overlaid steps based on IEEE 1730 and IEEE 1516

Full size image

European external border control activities nowadays strongly rely on human personnel and traditional manned assets such as aircraft, vessels or cars. The operational environment challenges the boots on the ground, which often need to operate in harsh conditions such as snow, high wind, rough sea state and deep mud. The diversity of the threats and the increased availability of low cost technological solutions pose a serious challenge for Law Enforcement and Border Agencies. All these factors make current operations both expensive and resource-intensive.

Significant technological advances have been made by European countries and companies in recent years, especially in the field of autonomous vehicles, to address the challenges mentioned above. Nonetheless, nations still lack solutions to address the complexity of the overall picture. The European Union and NATO are looking at integrating capabilities, creating complex architectures of multi-domain manned and unmanned systems and enhancing Command and Control by adopting Augmented and Virtual Reality. Within the framework of the ROBORDER H2020 project, the authors are involved in developing and demonstrating an autonomous border surveillance platform with unmanned multi-domain vehicles.

The resulting capability should satisfy tomorrow’s need for reducing the workload on border authorities and provide the ability to remotely sense, evaluate and track incidents at the borders. Early incident assessment based on enhanced data acquisition means a safer operational environment for the deployed human personnel.

The complexity of the solution may require additions to traditional design processes. There is also a need of assessing how the design of such a heterogeneous solution meets the expectations and operational requirements of the customers and users. Modelling and Simulation (M&S) is the methodology used to support all phases of the life cycle of systems, system of systems and concepts.

Until few years ago M&S has been used for ROBORDER like case studies to support the evaluation of the dynamics of specific vehicles or specific sensors performances or algorithms [2, 3], while only recent trends showed the application simulations of complex multi-domain heterogeneous surveillance systems [4] with the inclusion of realistic weather models [5,6,7] for supporting the evaluation of systems of systems performances.

In this project, simulation has been adopted to demonstrate its potential as a tool to evaluate system performance in large-scale scenarios with realistic operational conditions. In addition, an M&S-based testbed capability, developed since the early stages of the architecture lifecycle allows de-risking the design, while the integration of interoperable simulation with a platform permits the analysis of the algorithms developed to support future real operations. In this first iteration, simulation has fulfilled all these goals by supporting the verification and validation of algorithms and software components developed by technical partners. To demonstrate the platform adaptability, a set of Pilot Use Cases (PUCs) provided by the end-users were identified to test the performance against a range of conditions. Each PUC represents sea and/or land scenarios where the ROBORDER platform will be demonstrated.

The M&S environment complies with High Level Architecture (HLA) interoperability standards [8] and is designed according to IEEE and NATO best practices. It consists of a federation of simulators including virtual and constructive components and is able to run both in real and fast time.

2 Simulation Environment

The areas of development required for the ROBORDER M&S capability models are elicited from the analysis of the project objectives and requirements, the ROBORDER components (platforms, sensors and algorithms) and the characteristics of the maritime scenarios addressed in this paper. The modules identified (Fig. 2) are described in the subsections that follow.

Fig. 2.
figure 2

M&S capability architecture

Full size image

2.1 Platforms

A set of platform models has been created with the aim of providing the simulation entities (Fig. 3) with kinematic behaviours, guidance and effect of the environment. Kinematic data are shared over High Level Architecture (HLA) interface and consumed by the others federates.

Fig. 3.
figure 3

ROBORDER platform considered for this paper

Full size image

Kinematic Model

Platforms have been grouped into 3 types, based on their kinematic model properties:

  • Hovering platform (Copting UAV): In the kinematic model, all axes (Fig. 4) can be actuated independetly of each other degree of freedom. There are no cross couplings between axes. Pitch and roll (axes 5 & 6) motions are assumed to be minimal.

  • Torpedo platfom (Oceanscan-UUV): Yaw and pitch (axes 4 & 5) motions are a function of forward speed (axis 3). Vertical speed is a function of forward speed and pitch angle. Heave and sway (axes 1 & 2), as well as roll motions (axis 6), are assumed to be minimal.

  • Fixed wing aircraft (Evads-UAV): Vertical position is a function of forward speed and pitch angle. Yaw and pitch (axis 4 & 5) motions are a function of forward speed (axis 3). Yaw rate (axis 4) is a function of forward speed and roll (axis 3 & 6). Motion along vertical and lateral axes (axis 1 & 2) are assumed to be minimal.

Fig. 4.
figure 4

Platform kinematic model (left handed) axis set

Full size image

The kinematic model includes the physics of the platform and its low-level control system, merged into a single mathematical model. The control system allows the platform to follow target commands of speed, heading and altitude/depth.

For all three-platform types, the longitudinal speed and pitch degrees of freedom have been implemented as independent, second order functions. An additional independent, second order function has been implemented for the heading (hovering and torpedo platforms) and roll (fixed wing platform). Both Torpedo and Fixed wing asset types assume forward speed is always greater than zero.

The equation and the block diagram detailing these transfer functions are depicted in Fig. 5 with the following characteristics:

  • Dumping: Critical dump to avoid overshooting while following the target command;

  • Settling time: validated with actual telemetry data;

  • Saturation: In order to avoid exceeding the maximum rate of change of each variable, saturation values have been added.

This approach allows modelling the systems in a straightforward way, relying only on interactions with system providers and the limited data that can be inferred by the telemetry.

Fig. 5.
figure 5

Second order saturated system

Full size image

An additional step has been done for the altitude/depth command. Since the vehicles are able to change depth only by adjusting pitch angle, a PID closed-loop control system (see Fig. 6) has been developed around the pitch degree of freedom.

Fig. 6.
figure 6

Altitude/depth feedback control loop

Full size image

Platform Guidance

The guidance module is in charge of providing the kinematic module with the proper steering commands to perform the assigned mission. The steering commands are:

  • Target forward speed

  • Target heading

  • Target altitude/depth

Once the kinematic module receives the reference steering commands from the guidance module, it is able to follow them, since it implements both the physics and the required low-level controller.

The guidance module is able to handle waypoint missions: the mission is a sequence of waypoints each one having a set of common parameters like: geographical position, altitude to be reached, speed to be kept, next waypoint, etc. This mission is supposed to be accomplished moving in straight line from one waypoint to the next one. However for each waypoint it is possible to specify how to link two consecutive straight lines: passing over the specified waypoint and thus overshooting the transition to the following (fly-over) one or anticipating the transition before reaching the waypoint and thus shortcutting the path and linking smoothly to the next straight leg (fly-by).

Each kinematic model is implemented to take into account sea current (for marine vehicles) and wind. The model is able to compute the speed and course relative to the air/water, Speed over Ground and Course over Ground.

The model has two working principles to compensate the cross track error, based on the navigation sensors on board the vehicle:

  • The vehicle estimates its position (e.g. INS): the guidance sets the heading to the direction of the next waypoint, this behavior is implemented for underwater platforms (Fig. 7a).

  • The vehicle has feedback on its position (e.g. GPS): guidance compensates cross track error instant by instant, this behavior is implemented for fixed wing and rotary wing platforms (Fig. 7b).

Fig. 7.
figure 7

Guidance models implemented to compensate for environmental effects

Full size image

Further environmental effects relate to the limited conditions for Launch and Recovery of the platforms. Operational limit conditions are inferred from surveys with vehicle providers. For instance, the UAV provider states that the platform is unable to operate if the wind speed is higher than 20 m/s. This will trigger a mission abort in case the limit is exceeded.

2.2 Sensors

A set of sensor models has been created with the aim of providing the simulation sensors (Fig. 8) with field of view, detection and classification performance and effect of the environment. Detection and classification data are shared over High Level Architecture (HLA) interface. These data are used as an input to models from across the federation.

Fig. 8.
figure 8

ROBORDER sensors considered for this paper

Full size image

Two models have been developed to compute detections and classifications. The first model reproduces directional (frustum shaped) or omnidirectional (sphere shaped) field of view, using colliders in Unity 3D. This module is also in charge of providing the information for the computation of the sensor coverage of the mission area. The second module is in charge of simulating the performance of the detections and classifications algorithms. The process is triggered when targets are within a sensor’s field of view. The detection and classification module is out of scope for the experiment described in this paper and hence its description is not included.

2.3 Environment

The environment is divided into four gridded zones, the Air Column, the Water surface, the Water Column and the seabed. Each zone is broken into a series of ‘data cubes’ that contain all of the relevant environmental attributes. These cubes are referred to by row, column and, in case of the air and water columns, layer values (Fig. 9). The model contains publicly-available data sets that allow historical and forecast environmental conditions to be represented.

Fig. 9.
figure 9

Environmental model database approach

Full size image

2.4 Autonomous Resource Task Coordinator Integration

The Autonomous Resource Task Coordinator (ARTC) module is responsible for designing the trajectories for all the operational vehicles, having as objective to cooperatively cover (map + monitor) a user-defined region of interest (ROI). Before the trajectory calculation, the module incorporates the following user-defined parameters:

  1. 1.

    The ROI to be covered in a form of polygon in WGS84 coordinates

  2. 2.

    Areas (inside the previously defined ROI) that should be neglected from the mission. These areas could correspond to subparts of the ROI where i) autonomous operation is not allowed (e.g. actively covered by personnel), ii) the morphology of this area is not suitable for autonomous operation (e.g. presence of obstacles), or iii) the underlying information is not important/already known (e.g. large empty spaces). Therefore, any operation inside these zones would be either unsafe or a waste of resources.

  3. 3.

    The number of vehicles that are going to be deployed along with their operational capabilities (e.g. battery level, maximum flight time, etc.) and their initial positions.

  4. 4.

    The required level of details in the acquired sensors’ readings that is expressed with the scanning density parameter. In simple words, scanning density denotes the distance between two sequential sweeps.

As first step, ARTC discretizes the ROI into a number of grid-cells based on the user-defined scanning density. Special attention has been paid to properly place (position + orientation) these cells inside the ROI, having as objective to maximize the part of ROI that will be covered, if one just visits these cells. Based on this idea, an optimization process utilizing the Simulated Annealing [9] algorithm is proceeded, to maximize an optimization index that is directly connected with the coverage of the ROI (Fig. 10).

Fig. 10.
figure 10

ARTC functionality example

Full size image

Having an optimal representation of the ROI on grid, DARP algorithm [10] takes over to carefully design the trajectories for all the autonomous vehicles so as i) all the grid-cells are covered, ii) all autonomous vehicles are efficiently utilized, iii) without any unwanted overlap, and, iv) with the minimum possible number of turns that slow down the mission and increase the energy demands. It should be highlighted that, the service does not provide “blindly” equal paths for all the autonomous vehicles, but incorporates each vehicle’s characteristics (3rd bullet from the user-defined parameters) to produce paths tailored to their operational capabilities. Finally, the paths are designed in such a way to be ready for execution not only once, in a case where just a “snapshot” of the operational area is needed, but also in a continuous manner, in a form of patrolling the whole area, capable of monitoring the evolution of dynamically-evolving phenomena.

Such an energy-aware design of paths, but most importantly the efficient utilization of a team of autonomous vehicles, can be of paramount importance in the time-critical applications of ROBORDER, where complete mission awareness is needed as soon as possible.

3 Scenario

In ROBORDER, the scenario was developed in close collaboration with project partners. The development of the scenario was carried out in an iterative process, using NAF L2-L3 (scenario overview in Fig. 11) and L6 (logical sequence of events and interactions in Fig. 12) views to aid communication between communities.

The scenario, located in the bay of Thessaloniki in the Aegean Sea, has been created together with CERTH to provide quantitative means for assessing the performance of the mission planner, supporting the V&V of the ARTC. The scenario includes Fixed wing UAVs and UUVs deployed in a coverage mission.

Fig. 11.
figure 11

NATO architectural framework L2-L3 view of the scenario

Full size image
Fig. 12.
figure 12

NATO architectural framework L6 view of the scenario

Full size image

4 Results

4.1 Simulators Verification and Validation

The Verification and Validation of the models developed for the federation was conducted via a series of test runs specific for each model.

Platform

Assets federate validation was performed by comparing the real telemetry data with simulated trajectories. The platform providers were asked to provide a mission executed by the vehicle and the relative telemetry as ground truth. The mission was executed by the simulated asset.

For the fixed wing asset type, the validation of the kinematic model has been done with the Atlantic I vehicle, provided by EVADS. The choice of the asset was made based on the completeness of the dataset provided by the assets provider; the one for Atlantic I was the most suitable for validating the kinematic model. Fig. 13 shows the ground truth (green) and the simulated trajectory (red) for the Atlantic I. Table 1 shows the error in the simulated trajectories; the performances of the models are evaluated computing:

  • Maximum error: projecting the trajectories on the horizontal plane, the maximum length of the segment orthogonal to the ground truth intersecting the simulated trajectory.

  • Average error in the curve: average distance (computed as above) measured where the model has the lowest accuracy (the curve between two straight paths).

  • Average error over minimum mission legs distance: the ratio between the error approximating the curvature radius and the minimum curvature radius in a mission.

  • Mission duration error: the error between the duration of the real and the simulated mission.

EVADS assessed that the asset simulator is very accurate reproducing the behaviour of the platforms, validating the asset simulator. Figure 13 shows the ground truth (green) and the simulated trajectory (red) for the Atlantic I.

Fig. 13.
figure 13

Asset validation for the Atlantic I vehicle, in perspective (left) and from above (right) (Color figure online)

Full size image
Table 1. Errors in kinematic model
Full size table

Torpedo asset type model is based on CMRE accredited models. The model was previously validated using vehicles similar to the LAUV, specifically the Bluefin Muscle; it is thus assumed that no further validation is needed for the torpedo asset type.

Sensors

The sensor models are based on the reproduction of behaviours based on configuration files customizable by simulation users.

The key activities in the test of the model have been verification tests to ensure that the dimensions representing the fields of view have a 100% match with the values provided by the technical partners.

Environment

The meteorological and oceanographic model was based on the representation of previously validated, standardised datasets.

With this in mind, the key activities in the test of the model were the verification tests to ensure that the imported data values matched those provided to the distributed simulation environments.

4.2 Experimental Set up for the ARTC

The experimental set up included two sets of missions for testing UUV and UAV missions. The main objective was to evaluate the feasibility of the missions, measuring the intersections that may occur while following the simulated trajectories, because of the way UxVs pass through turning points.

For the first set, five different missions were generated (Table 2). All of them are utilizing five UUVs, however with different scanning densities, from 50 to 300 m and different turning modes (fly-by/fly-over).

The second set of missions (Table 3) involve fixed-wing UAVs. These simulations were helpful in order to decide the best platform guidance to employ, the number of UAVs that could, or should, be utilized to cover an area and the appropriate distance between the designed trajectories (scanning density).

The evaluation of the performance for both sets is obtained through both the visualization of the trajectories of the simulated assets (example in Fig. 14) and from the analysis of the values of a set of performance indicators, down-selected from project’s KPI set. The KPIs used are:

  • KPI_1: Intersection. The number of times two platforms are violating the safety distance between them.

  • KPI_2: Intersection duration. For each intersection, the time in seconds the platforms are violating the safety distance between them.

  • KPI_3: Mission length. The amount of time in hours it takes each vehicle from the start to the end of its mission.

  • KPI_4: Mission coverage. The percentage of the mission area covered by the sensor of each vehicle.

Table 2. UUV simulations
Full size table
Table 3. UAV simulations
Full size table
Fig. 14.
figure 14

Example trajectory of two simulated UAVs (left) and five UUVs (right)

Full size image

4.3 Discussion of the Results

The analysis of the results of the simulations has been performed checking the values of the KPIs and investigating the trajectories of the simulated assets; the analysis showed the following:

  • Trajectories intersections: Intersections may happen during transit from the launching location to the first mission waypoint. It is suggested to carefully identify the launching position based on the portion of the mission area covered by each vehicle or in case this is not possible to shift the launch time in order to avoid them. The simulations also identified that intersections might happen during the execution of the missions. This could happen for missions that were designed with no respect for the kinematic limitations of the involved vehicles (e.g. minimum turn capability). Simulation ca be used to identify missions with collision risk and reject them, as they are not safe to be executed in real world scenarios.

  • Mission duration: none of the results suggested that nominal vehicle autonomy was exceeded.

  • Mission coverage: simulation results identified that the mission area is completely covered, regardless of the distance between mission legs. This suggests that portions of the mission area are redundantly covered. Further investigation on this aspect showed that redundant coverage seems to be caused by unrealistic assumptions on both sensors field of view and vehicles trajectory computation within the ARTC. In order to shine a light on this phenomenon, it was suggested to include an additional KPI computing the percentage of the mission area that is covered more than once, as well as a visual representation of this phenomenon. This should allow CERTH to identify the optimal distance between mission legs, minimizing the redundant coverage.

5 Conclusions and Way Ahead

The paper describes the design and use of the M&S-based test bed capability developed for the H2020 ROBORDER Project. The work done demonstrates the potential of M&S as a tool to evaluate system performance in large-scale scenarios in realistic operational conditions. The M&S environment complies with High Level Architecture (HLA) interoperability standards and is designed according to NATO best practices.

The developments described were driven by the maritime scenario developed to support the Verification and Validation of the Autonomous Resources Task Coordinator, a mission planning and management software module developed by CERTH, for safe and efficient autonomous missions. The outcome of the research showed that simulations can significantly help to adjust the missions generated by the ARTC in order to achieve the desired coverage, without wasting operational resources, in realistic conditions that will be faced in a real-life experiment.

The results also identified areas of further development, i.e. the inclusion of KPIs for computing the coverage redundancy and the visualization of area coverage on a map for each vehicle. CMRE team is currently working on those aspects and new results are expected to be included in follow-up simulations.

CMRE team is currently working on enabling project partners to configure and run their own specific scenarios and support dedicated experiments for their specific goals.

Next steps involve the implementation of the feedback provided by CERTH to further study the performance of the ARTC module for both test scenarios and on specific scenarios that will be demonstrated in the project live demonstrations in Portugal, Hungary and Greece.