Keywords

1 Introduction

EWISA project was the result of a call for proposals restricted to a consortium of National Border Authorities from Greece, Finland, Spain, and Romania. The 58-month project’s objective was to provide an operational and technical framework that would increase situational awareness and improve the reaction capability of authorities surveying the external land borders of the EU. EWISA provided an innovative system for warning about possible threats for all border control relevant systems, equipment, tools, and processes for the surveillance in selected areas.

EWISA promoted further cooperation among public authorities in charge of surveillance of selected parts of the external EU land borders, so as to improve the quality and competence of their services (as related to security), through the Pre-Operational Validation (POV) concept for novel solutions.

In the context of EWISA, for the first time, four EU MoI (Ministry of Interior) Authorities jointly determined defined:

  • A vision to improve overall situational awareness

  • A common concept – the same core technologies for all areas of implementation

  • The validation strategy

Pre-Operational Validation process provides a tangible assessment of the performance levels offered by innovative technologies in a realistic user-defined operational scenario, where a trade-off between efficiency, effectiveness, and cost can be aligned with actual needs.

2 EWISA Core System

EWISA concept is based on the development of a flexible, modular surveillance capability which maximizes the use of existing sensor types, including both static and mobile/deployable sensor platforms, following the concept of a unified integrated solution for the external EU borders based on data fusion from heterogeneous sensors, including Video Analytics Technologies generating intelligent analysis reports (Fig. 14.1).

Fig. 14.1
figure 1

Land border typical solution

Full size image

The common core of the project was the development and the validation of the video analytics and data fusion components which were represented as Centralized in National Coordination Center (NCC) level and Decentralized in Command Center/Regional Command Center (LCC/RCC) level. Other sensors or sources at the national or regional level were also integrated within the core system in order to support the proof of concept of EWISA.

The objective was to increase intelligence in surveillance both in a qualitative and quantitative manner. The project provided an innovative system for warning on possible threats, enhancement of effectiveness and efficiency of all land border control relevant systems, equipment, tools, and processes for the surveillance in the selected areas.

The core of the EWISA system is introduced in Fig. 14.2 with the two fundamental components which are:

  • Video Analytics Component (VAC) fed by video sources

  • Data Fusion Component followed by Intelligence Analysis Reporting fed by the VAC information and the input of other Surveillance Supporting sensors/systems such as radars and ESM (electronic support measures) installed on stationary or mobile platforms

Fig. 14.2
figure 2

EWISA core system flow

Full size image

In order to achieve a practical implementation of the EWISA core system, other supporting equipment had been deployed along with video analysis components, either to facilitate the provision of coherent inputs to the VA system or just to guarantee adequate performance of the overall surveillance deployment. The supporting surveillance equipment included in the EWISA project consisted of the following:

  • Land Vehicle with EO/IR/SWIR/RF/SL/LP

  • Low Emission Radars

  • ESMs

  • Fiber Optics system

  • Boat with EO/IR/SWIR/RF/SL

  • Aerostat with EO/IR/SWIR/RF/SL

EWISA did not deal with stand-alone technology providing new capabilities. It rather validated (in terms of capacity to meet the requirements set by the public authorities) the integration of novel solutions, proposed by technology developers, into the current/legacy surveillance infrastructure. The realization of the aforementioned setup was through an innovation procurement procedure which concluded with two successful tenderers developing their own technical solution approach.

3 EWISA Validation Methodology

The consolidation of a concrete validation strategy that could be utilized also for other similar testing activities was one of the core activities of the EWISA project. The outputs of the validation process of the offered solution had to ensure that the partners and other stakeholders in land border surveillance framework to:

  • Check that the implementation process followed by the contractor had been correct

  • Measure the level of compliance of EWISA solutions with the partners’ operational objectives

  • Compare two different alternatives based on different aspects of interest such as performance, deployability, operational value, etc.

In this sense, the validation process was based on being able to provide answers to the following questions:

  • How does the EWISA solutions perform? (answered by testing the technical performance of the solutions)

  • Does the EWISA solutions fit to End User’s expectations? (answered by measuring the user acceptance of the EWISA solutions over a group of users with responsibility in land border surveillance)

Thus, the EWISA concept of validation, as it is depicted in the following Fig. 14.3, proposes an assessment of the solutions from two complementary perspectives: technical and operational. A third interesting perspective that needs to be taken into account is the cost analysis which is not going to be analyzed in the context of this chapter.

Fig. 14.3
figure 3

Validation strategy phases

Full size image

3.1 Technical Verification

The technical verification consisted in checking if the contractor has “built the solution right.” This was realized through the following means:

  • Monitoring the progress in the development of the technical solutions towards providing the R&D service

  • Measuring the level of compliance of the EWISA solution with the system requirements and performance levels

This technical verification was ensured throughout the project in the form of continuous monitoring activities and visits on the contractors’ premises. In addition, the level of compliance was measured in EWISA project in three different stages:

  • LAT (Laboratory Acceptance Tests): This verification is the natural first stage at every development. Each company carried out their own tests prior to step towards the next phase of the deployment.

  • FAT (Factory Acceptance Tests): New tests, both modular and as a whole system, were carried on in the companies’ facilities by themselves, but under the supervision of the EWISA Consortium this time. Their objective is to check if every requirement is properly fulfilled and the maturity level of the system is high enough to go ahead with the on-site deployment and the integration with the legacy systems of the End Users.

  • SAT (On-Site Acceptance Tests): This verification was performed during the deployment of the EWISA solutions. The EWISA Consortium checked that the solutions delivered meet the technical specification by supervising the verification procedures that were carried out by the industry developing the solutions in every scenario, as a part of the scope of the contract.

For the last two tests, EWISA followed the below-mentioned stepwise methodology:

Step 1. Identification of Requirements

The longlist of the requirements was categorized using the MOSCOW method based on their relevance and importance on each test. It was commonly decided that the “Would” will not be checked in the FAT neither the SAT.

Step 2. Identification of Test Scenarios

The test scenarios were initially built during the start of the project; however, a refinement was performed before each testing procedure in order to ensure their alignment with the testing objectives. The reference scenarios were used to experiment variations along the contract execution derived from factors such as changes in modus operandi of different targets, the availability of assets or modifications in the area of interest.

Step 3. Identify Team

For both tests the team was comprised by one technical representative of each partner, with the support of experts.

Step 4. Preparation for Test

This step has been mainly undertaken by each contractor before the start of each testing activity.

Step 5. Run Test and Track Results

The scenarios were executed, and the technical representatives were requested to assign a successful or not verdict to each requirement.

Step 6. Checking Whether the Requirements of the Customer Are Accomplished by Analyzing the Verdicts

Following the test execution, each member of the FAT team assigned a grade of severity (Step 5) to all defects identified on the requirements and the scenarios included into the checklists during the test execution. Based on their classification in Step 1, the requirements with defects were placed on a classification table.

3.2 Operational Validation

The operational validation is about answering the question “Did we provide the right service?” In other words, it consists in evaluating whether the service to be delivered meets the End Users expectations. With this aim, an operational evaluation process was set up in EWISA to be executed during the operation stage.

End Users as main beneficiaries of the technologies were responsible for validating the solutions built and tested by the industry. This operational validation was deemed necessary because a correct technical implementation compliant to requirements does not necessarily imply a high End User satisfaction. In some cases, the operational needs are not translated accurately into the technical specification, and thus the solution built does not provide to the End User the operational added value expected. The same solution may have a different operational value for each End User and scenario, so it is needed that the End Users have a tool that allows them to measure the operational value that the solutions add to the execution of their operational tasks in their real scenario.

The validation procedures established for EWISA project fulfilled the following conditions:

  • Flexible enough to be adapted to the needs of EWISA project. Some factors as changes in operational environment derive in changes in the validation needs and though processes should be easily adapted to changes in context. These types of changes could imply to upgrade the type and content of the information to be gathered or the way to gather it.

  • End Users must understand what they measure within each indicator in order to obtain added value objective results. In this sense it was important to ensure that the MoEs and the metrics used to evaluate were interpreted equally by the whole community of End Users and no ambiguity existed when providing the measures. Thus, training sessions have been imparted in order to unify End User’s criteria and solve doubts.

  • Measurements must be effective. The results obtained after analyzing the information gathered should help decision-makers to understand project issues and to evaluate services aspects such as performance, costs, or maturity. Obtaining useful measurements requires the fulfillment of the two previous statements.

The operational validation comprised the following activities:

  • Planning validation: comprised all the activities for launching the validation process.

  • Information gathering: End Users gathered information during the operation of the EWISA Solutions and evaluated the indicators according to the information gathered.

  • Processing the evaluation provided by End Users and generated conclusions.

3.2.1 Definition of Validation Concepts

The operational validation process was, therefore, devoted to determining at what measure the EWISA solutions complied with End User’s objectives, ensuring that they fulfilled the requirements established in the terms of reference from an operational standpoint. This validation strategy was, therefore, sustained on a validation taxonomy built upon aggregated measures which addressed the effectiveness of the developed solution. This taxonomy was composed of five main concepts that expound on the following sections:

  • Operational Obstacles: the main difficulties detected in the different scenarios that complicate the detection and prevention of illegal activities at the border.

  • Key Performance Area (KPA): areas defined as most important in determining whether a system has been improved by a new operational measure.

  • Key Performance Indicator (KPI): critical subset of performance parameters representing the most critical capabilities and characteristics.

  • Measures of Effectiveness (MoEs): These KPIs will be composed of measures of effectiveness (MoE) intended to provide a measure of the expected systems performance in the operational environment according to what the End User expects.

  • Metrics: These MoEs, likewise, can be further broken down into metrics when necessary, in order to increase the granularity of the measurement done on the system.

As a preliminary step, prior to the definition of the metrics to evaluate the operative value of the solutions, it was necessary to define the Operational Obstacles faced by the different End Users in their daily work within the framework of border surveillance. The goal is that, once the validation process is finished, it will be possible to determine to what extent the technical solutions of EWISA have contributed to overcome the operational obstacles defined by the End Users. The validation strategy shall allow End Users to measure at what level their expectations have been fulfilled. With this aim, the End Users have translated their expectations into a set of operational objectives which are the indicators to be evaluated using the operational validation process. For the purpose of the project, the operational objectives have been defined using the requirements, capabilities, and the principles of the Concept of Land Border Surveillance established.

The operational objectives have been classified into six key performance areas (KPAs) which are the areas of capability to be reinforced through the solution under validation in order to increase the operational effectiveness of land border surveillance. The KPAs match with the six areas of capability for classifying the system requirements. A number of operational goals were structured into a set of key performance areas. Each area comprised a set of capabilities that led to an improvement of the operational value of the solution.

KPA 1: Command, Control, and Coordination

This area comprised a set of capabilities for improving command, control, and coordination during the operations at different operational levels (tactical, strategic). The main capabilities were related to support planning and decision-making through an enhanced situational awareness and an efficient use of the resources. This Capability Area directly related to the project objective “Achieve a high level of control” had a multiplying effect as it maximized the effect of the rest of the capabilities.

KPA 2: Acquisition

This area comprised capabilities for improving the detection, monitoring, and identification of targets of interest in land borders through the acquisition of more reliable and precise information. The acquisition of information from external sources such as new sensors/platforms, open sources, or external DDBB were also considered. This Capability Area was directly related to the project objective “Detect irregular movement,” “In-depth observation/identification.”

KPA 3: Exploitation and Analysis

This area comprised a set of capabilities for fuse, correlate, process, and exploit the information acquired from different sources (sensors, platforms, external systems) to generate intelligence from the raw data acquired.

KPA 4: Communications

This area comprised capabilities for the well-dimensioned, robust, and secure transmission of data between the different assets/centers involved in land border surveillance in order to allow the availability of the necessary information at the precise moment and location.

KPA 5: Mobility and Projection

This area comprised capabilities for disposing of the necessary means in order to allow strategic deployment and high mobility of assets and personnel as required by the operations. The objective is to allow the intervention in the area of interest at the required moment.

KPA 6: Sustainability

This area comprised capabilities for guaranteeing the sustainability of the resources in the area of operation during the mission.

Figure 14.4 shows the classification of EWISA operational objectives into areas of capability (or KPAs). The EWISA solutions provided new and/or enhanced functionalities to improve the operational value of the solution perceived by the End User in one or more KPAs, contributing to the consecution of one or more operational objectives. Moreover, KPIs were used to measure the level of improvement provided by the solution on each key performance area (KPA). These measures described how well a solution achieved its objectives. They were the critical subset of operational performance parameters representing the most critical capabilities and characteristics in each particular area, and, of course, they excluded the evaluation of the performances of the legacy systems.

Fig. 14.4
figure 4

Classification table

Full size image

4 EWISA Operational Validation Execution

Following the definition of the abovementioned methodology and metrics, both solutions were deployed in four diverse geographical areas of EU external land borders (Figs. 14.5 and 14.6) as agreed by the consortium of EWISA.

Fig. 14.5
figure 5

EWISA operational objectives by KPA

Full size image
Fig. 14.6
figure 6

Phase 3 solution testing

Full size image

In this regard, the validation of the solutions from an operational perspective was performed for an 8-month period in a real environment in surveillance operation. The EWISA concept was tested in a real operational environment, based on well-defined scenarios, representing the EU external borders environment and concept of operation, as follows (Fig. 14.7):

  • Greece: Surveillance of north area of Evros River in open and semi-open area, on the borders with Turkey.

  • Finland: Surveillance of border line and border opening combined with surveillance of border zone boundary in a forest area, uneven, or rough land with Russia.

  • Spain: Surveillance of the border line in Melilla area with Morocco.

  • Romania: Surveillance of the border line with Serbia at terrestrial border and along the Danube River.

Fig. 14.7
figure 7

EWISA test sites

Full size image

This validation provided a qualitative measure of the level of compliance of the EWISA solution with respect to the End Users’ objectives. Moreover, the operation of the solutions in real environment allowed the End Users to measure the abovementioned set of operational indicators (or measures of effectiveness) defined by the EWISA Partners (End Users).

So, the solution developed by the contractor was integrated in additional legacy systems into real operation conditions. During this stage, the contractors ran the solution that integrated the EWISA concept, as developed and tested in previous stages for 4 months at each test site. Each scenario had its own schedule and ran for a certain amount of time during that frame.

After the first half of the operation time, the End Users reviewed the operation status and the intermediate results in order to determine if there were any deviations from the expected deployment by conducting a technical verification, the SAT, as described previously. When the solutions successfully passed all tests meaning that they were identified to run as expected, the solutions continued their operation without interruption to each site.

5 EWISA Operational Validation Results

In continuation to the above, an online survey was organized for the End Users. The validators have received a link to participate in the validation survey, and they decided which module to evaluate, according to their operation scope in EWISA project. Each evaluator filled a set of MoEs depending on their role and location from which they have been operating.

They had to evaluate the applicable metrics for each MoE, scoring the performance brought by the solution using a 1–5 scale:

  • 1: the solution provides very poor performance (it is not implemented or it is not operative due to serious malfunctions).

  • 2: the solution provides a slight performance.

  • 3: the solution provides acceptable performance.

  • 4: the solution provides considerable performance.

  • 5: the solution provides great performance.

Besides, for each metric, the validators were also able to say how it has been the EWISA experience compared with “outside EWISA” systems (e.g., existing legacy systems), if any. This was very important also for the evaluation because this way the Consortium could assess not only if the systems performs well, but also if it has provided added value to the End Users. In this case, the evaluator had to check a higher, equivalent, or lower performance of EWISA with respect to outside EWISA systems. The N/A option was also available.

Based on the input provided by the evaluators, a summary of the results per solution has been created, as presented below (Figs. 14.8 and 14.9):

Fig. 14.8
figure 8

Solution 1: General statistics

Full size image
Fig. 14.9
figure 9

Solution 2: General statistics

Full size image

Both solution’s overall results have been positive. The validation process has been globally performed as it was planned in each prescribed phase. An interesting comparison between the delivered solution and the current used system has been performed, demonstrating its added value with the respect to the state of the art. Results have been assessed and discussed, showing an adequate satisfaction of user needs in both solutions.

6 Conclusions

Considering that the core element of the EWISA project is the POV of technological solutions in an operational real scenario, End Users were heavily involved in all procedures. Despite the inherent complexity of the EWISA project, the operations of both solutions were implemented and experimented in real environmental scenarios. In general, the developed technologies delivered outstanding added value in the scope of the border surveillance.

More specifically, the LE radar was considered as an important asset to the technical surveillance capacity. For the radar sensor, target detection, tracking, and separation performance was sound for all tests, with highly accurate, near real-time results. Radar coverage was continuous with high quality and reliability. However, it is true that in some cases, the detection of people by radar is difficult due to the proximity (less than 50 m) of roads and houses with a massive influx of people on the surveillance area.

Demonstration offered also a good opportunity to test the video analysis features. In this case, the detection of movements of groups through video analysis resulted more effectively in the short range. Additionally, it was considered that the new sensor could work also as an important deterring element helping the End Users to perform their daily activities.

Regarding the ESM, it was considered of high value for the surveillance activities. The FO sensor was also considered interesting new technology which could enhance the performance of the technical border surveillance. The sensor was able to detect single targets, distinguish between simultaneous targets, as well as split groups to individual targets, performing within the required range and accuracy specifications.

From the operational point of view, the integration of terrestrial radar systems and different sensors provided new capabilities for border early warning.

The implementation of this project introduced some innovative features such as (i) the diverse environments where all demonstrations took place, covering many different environmental setups, (ii) long-lasting demonstrations of 8 months, and (iii) frequent presence of End Users throughout the 8-month period for validating the solutions and infusing insights and feedback valuable for delivering accurate results.