Keywords

1 Introduction

EL-Vs are a further step towards an even more sustainable urban mobility. The Users’ Low Awareness about the EL-Vs performance and functionalities has been acknowledged as one reason hindering the EL-Vs market penetration and it is partially due to the limited direct experience of such vehicles and the consumers’ concerns about the relatively high cost of such vehicles and the feeling of uncertainty as regards the possible need to re-charge the vehicle during the trip. Additionally, there is a lack of consistent knowledge and information needed by planning authorities to prepare an adequate traffic and charge infrastructure to achieve their integration in the transport and electricity networks. For example, little is known on how such vehicles are used and how much basic infrastructure is required in terms of parking spaces and type and location of charge facilities [1]. ELVITEN project [2] organizes long demonstrations of EL-Vs in 6 EU cities to demonstrate the potential market penetration of EL-Vs. The main objectives are to enhance users’ awareness about EL-Vs and to collect data in order to generate guidelines for manufacturers and authorities for the better integration of such vehicles in the transportation and electricity networks. In order to enhance the credibility and validity of the upcoming analysis results the methodological simultaneous triangulation concept has been exploited [3], [4] where qualitative and quantitative methods for data gathering are used concurrently and the findings complement one another at the end of the analysis. Following this methodological framework, the consortium partners collect and analyze, using inductive and deductive approaches [5], various types of city transport data, data from the real trips done by people with such vehicles and data relevant to the usage of the related services and tools. Opinions and attitudes of people towards the EL-Vs are also collected via specific questionnaires administered via related services and apps as well as via wide online surveys and interviews. The data will be processed within the framework of the project during the evaluation phase in order to analyze mobility behaviour, actual usage of EL-Vs, charging behavior, experiences and attitudes towards EL-Vs and the associated services that will be evaluated against specific KPIs. In order to accomplish the data collection, ELVITEN utilizes a harmonized, user-centred process for demonstrations set-up and operation to ensure also that the needs of the citizens who use the system services and tools are fully taken under consideration, thus motivating them also to use the EL-Vs during the pilot phase. Apart from the data collection process that is of utmost importance, the analysis of the mobility situation in the demonstration cities and consequently the definition and deployment of the usage schemes, the identification of users types as well as their recruitment constitute very important activities to the demonstration methodology. Furthermore, the design, implementation and verification of the ICT tools and apps are focused on the individual users and are a mix of automated data logging via loggers installed on the EL-Vs, via user-friendly apps and services and via direct surveys through questionnaires and interviews. A design of the appropriate means (what type of survey, what questions to ask) to collect meaningful feedback from users has been conducted. Finally, the usage and data collection monitoring of the demonstrations is done via custom designed visualization tools (Fig. 1).

The description of the demonstration methodology followed in ELVITEN project constitutes the focus of this paper. The remainder of this paper is structured as follows: Sect. 2 presents the methodology for the analysis of the current situation in each city, to be used as reference baseline in the analysis and evaluation phase, Sect. 3 describes the procedures to involve and engage users and stakeholders, to get users’ consent to the processing of their data and to sign agreements with owners and long-term sharers of such kind of vehicles while Sect. 4 describes the demonstration set-up activities as well as the data collection and demonstration monitoring tools. Finally, Sect. 5 gives the conclusions.

Fig. 1.
figure 1

Demonstration methodology and data collection.

Full size image

2 Baseline Analysis Per City

For assessing changes and impact of ELVITEN, as it establishes a basis for comparing the situation before, during and after the ELVITEN project, a baseline analysis carried out to be a useful benchmark for examining what level of change is triggered by the project and can be also a crucial element in research and planning, besides in any monitoring and evaluation activities [6]. The data used in the baseline calculation is based on information provided by cities relevant to the current status of mobility: number of vehicles, typology, and other. The baseline values can be considered as benchmarks/starting values for the Agreed KPIs and can be used to analyse changes over time, by measuring and/or comparing values during and after the demonstrations. The Agreed KPIs are selected by asking the six cities to assign a numerical score, based on the level of importance of the indicator (with a scale from 0 to 10; 0 not important and 10 necessary). An initial screening included the selection of Agreed KPIs with scores greater than 8. For each Study Question (SQ) the number of KPIs that satisfied this parameter are identified; if no KPIs satisfied the first selection criterion for the Study Question, the KPI with the highest score is selected. Subsequently, the average values of the score assigned by each city are evaluated in order to choose the same KPIs for the cities. Afterwards, the baseline mobility situation for the Demonstration cities has been investigated in order to analyse the mobility conditions, infrastructures and EL-Vs present before the demonstration phase.

The last step of the adopted methodology for the baseline analysis was the calculation of baseline values of the Agreed KPIs. In the literature, different methods described how to set a baseline value. Sensible and useful baseline values can be obtained through enough performance measure values calculated [8]. Other studies estimate the baseline values using other techniques [8]. The methods to set the baseline values are related to the maturity of the KPIs: in some cases, it is possible to use historical data, in other it is needed to collect data in advance in order to set it or to use data estimation techniques. The common scenarios to set a baseline value for KPIs are (i) a brand-spanking new KPI with no data yet, (ii) a mature KPI with a lot of historical data and (iii) a mature KPI with a seasonal or cyclical pattern. In this study, as the selected Agreed KPIs are indicators without historical data, so they belong to the first category. More in detail, the calculation methodologies for each KPI are based on input data provided by users and EL-Vs during the demonstrations. The input data to calculate the baseline values are collected from the surveys conducted at the beginning of the project and information provided by the cities and the Agreed KPIs are the same for all the cities.

3 Users’ and Stakeholders Involvement

3.1 Usage Schemes and User Types

One of the main targets of ELVITEN project is to develop and demonstrate usage schemes for EL-Vs in six demonstration cities. At first stage a study related with the mobility situation in each city has been conducted and proposed the most promising usage schemes and target user clusters for the best integration of EL-Vs into the existing transport and mobility network in each city. Mobility demand, user needs, opportunities and barriers to EL-Vs use were gathered as follows:

  • Statistical data regarding the transport networks and mobility usage in each city (geographical/socio-economic characteristics, transport infrastructure, trip data, modal split, travel costs, etc.)

  • Data from a large scale public questionnaire survey (approximately 7000 responses) aimed at all citizens in the six cities (and also answered by citizens in other European cities) covering current travel patterns, whether they would potentially use different types of EL-V, and perceptions of advantages and barriers to using EL-Vs.

  • Data from a more limited set of targeted interview surveys of fleet operators and professional drivers (mostly focused on delivery companies) on their current fleet use and the potential scope of switching some or all of their ICE vehicles to EL-Vs (including perceptions of advantages and barriers).

Based on these data analysis the following different usage schemes were identified: (i) Public short-term sharing scheme, (ii) Private (individual) ownership scheme, (iii) Delivery sharing & ownership scheme, (iv) Corporate sharing & ownership scheme.

Therefore the target user clusters involved in ELVITEN demonstration activities were identified in general as owners and sharers while the types of users related with the functional usage of EL-Vs during the demonstration activities taking place at each city were identified as Short-term sharers of EL-Vs, Long-term sharers of EL-V and Owners of EL-Vs. The involvement of stakeholders is also considered very important for the acceptance of EL-Vs, regarding their willingness to use fleets of EL-Vs for their personnel and/or to offer the organisation’s parking facilities for charging private EL-Vs. The type of stakeholders are light delivery fleet owners, rental and sharing companies, tourism service providers, planning and public authorities.

3.2 User’s Involvement and Recruitment Procedure

ELVITEN uses several ways to involve the participants. In more detail, ELVITEN uses the partners’ networks and their national members, local associations of users’ groups and established Regional Support Groups (RSGs) in all the pilot Cities. It also undertakes wide awareness campaigns in each City, starting well in advance the start of the demonstrations period, to attract the interest by a large number of users and stakeholders. Such campaigns are being undertaken via local press and media announcements, announcements in relevant local and national associations and municipalities, web sites and social networks. Posters at key locations in each City and leaflets are also employed for this purpose as well as advertisement of the incentives given by the Municipalities from the usage of the vehicles and ELVITEN ICT tools and services.

3.3 Getting Users’ Consent and Ethical Procedures

In order to get consent from the users, an Information Sheet has been prepared and ELVITEN users are asked to read it before giving their consent to be involved in the demonstrations and the main procedures implemented are the following:

  • All personal data collected will be treated as confidential and no actions will be undertaken unless the partners have the consent by the users (data subjects). The data handling procedures strictly adhere to the Regulation (EU) 2016/679 [9].

  • Users are asked to read the information sheet and sign beforehand a consent form on the terms of data collection, treatment and further use.

  • Users are provided with access to their data when requested and ultimately have the right to force the deletion of said data or withdraw their consent.

  • Personal data will not be in any case shared with or disclosed to anyone outside the research team.

  • Activities only include adult research participants who are healthy and fully able and capable to provide informed consent to their participation and who sign the informed consent form before any such activity.

  • Participation of users is strictly voluntary and they have the right to withdraw themselves and their data from any ELVITEN activity.

3.4 Agreements with EL-V Owners and Long-Term Sharers

Apart from the consent form, two more types of agreements are signed by some owners of EL-Vs and long-term sharers before joining the demonstrations. The scope of the agreements is to define the terms under which these users commit to the demonstrations. For owners, this entails the acceptance of using a trip data logger and the conditions when ELVITEN has the right to request it back, for example in case of inactivity, so that it can be given to another user who would be more active and thus could contribute more data. Similarly, for long-term sharers the agreement entails terms relevant to the provision of the equipped shared EL-V and the conditions when ELVITEN has the right to request it back before the end of the agreement period. In more detail, the agreements define the following issues: (i) Equipment that is given to the user, (ii) Duration of the agreement & Conditions to terminate the agreement, (iii) User’s and ELVITEN partners’ responsibilities & Handling of disputes.

4 Demonstrations Set-up for Data Collection and Monitoring

This chapter describes the steps that the demonstration cities and the ELVITEN consortium partners follow in order to collect the necessary data for the analyses and evaluations that will be conducted at a later stage of the project.

4.1 Verification Methodology

The first demonstration activities were the short pilot dry-run tests that have been organized in each City with limited numbers of participants, to verify that all services and the ICT tools operate smoothly and to report any issues for corrective actions before the demonstration start. The usage schemes tests gave quite focused and detailed user feedback in order to refine the schemes, the services and tools as needed, before the start of the open demonstrations. A template for dry-run tests was developed, organizing the verification activities into use cases to cover all aspects involved in the verification. The dry-run tests use cases covered the following aspects:

  • Preparation: included verification whether the City was prepared for the tests, including the availability of test users and EL-Vs, the presence of ICT providers, etc.

  • Inventory Management: to verify the existing EL-Vs inventory and the installation of inventory-management oriented applications.

  • User registration: to verify the coherence and storage of data for the registration of users in order to give them access to the ELVITEN services and to verify the information provided to users.

  • Booking & Brokering Services: to verify different aspects of the booking and brokering of resources (EL-Vs, Parking Spaces and Charging Points) by users focusing on real availability as well as ICT tools that support these services and the data exchange between the middleware and the ELVITEN Data Warehouse.

  • Operations: to verify the complete process for long/short term sharing of resources to a user, including questionnaires administration and tracking of data exchanges from cities, users and vehicles through the middleware up to the Data Warehouse.

4.2 Tools for Quantitative Data Collection During the Pilot Phase

The trip data (such as location, speed, acceleration, trip duration, etc.) are collected via the trip data loggers (EL-V Live Data) which are installed on the EL-Vs and via the ELVITEN ICT applications from where also the questionnaires are administered. In general the ICT tools deployed include registration services, a Brokering and a Booking service for EL-Vs, parking and charging points, an EL-V fleet monitoring tool and an Eco-Drive app, a Serious Game app and an Incentive Management Smart Card. All the data are transmitted and stored in the Data Warehouse via the ELVITEN middleware. The detailed description of the ELVITEN ICT platform is out of scope for this paper however its schema has been provided in Fig. 2.

Fig. 2.
figure 2

The ELVITEN ICT platform and data repository.

Full size image
Fig. 3.
figure 3

The ELVITEN dashboardall cities overview.

Full size image

4.3 Tools for Qualitative Data Collection During the Pilot Phase

Qualitative data (reports on experienced problems, acceptance of EL-Vs, services and ICT tools, attitudes, willingness to use, etc.) are collected via the questionnaires which are administered by the ELVITEN apps. The questionnaires contain yes/no and ranking questions (5-point Likert scale about user’s satisfaction), the user’s unique ID, as well as open comment sections and their content is related with the following aspects:

  • Background data related to demographics, mobility habits and EL-V experience.

  • Characteristics of the last trip with an EL-V regarding the ride, parking and charging.

  • Experiences / issues encountered during the last trip.

  • Attitudes towards EL-Vs & Willingness to use/acquire an EL-V.

  • Attitudes and opinions towards each service, ICT tool and policy in place in the reference city.

Users are asked to fill in the questionnaires at a predefined time plan according to the usage scheme and submit the questionnaires immediately after their completion. If these questionnaires are not submitted in time, the users are reminded to complete and submit them via each tool and application. For all types of users the Background Data questionnaire is filled in and submitted before the first trip along with the registration process. The Experiences / issues questionnaire is filled in and submitted after each trip, if an issue has occurred during the last trip. The rest of questionnaires (mobility change and attitudes towards EL-Vs and towards the ELVITEN services and apps) for short-term sharers and owners are being administered in four months intervals while for long-term sharers they are administered after the end of the sharing period (e.g. week). Additionally the service operators report on a daily basis on i) the local weather and ii) how many of the shared El-Vs are operational. Furthermore, a wide online survey will be conducted to the middle of demonstration period related with the problems the users’ are facing as well as to charging patterns in order to support the data collection process. Finally, a citizens’ wide attitudes survey will be conducted after the end of the demonstrations, similar to the one carried out at the beginning of the project, to analyse the differences in the wide citizens population attitudes and opinions regarding EL-Vs before and after ELVITEN demonstrations.

4.4 Tools for Demonstrations Monitoring

During the demonstrations, mechanisms to effectively monitor usage such as number or registrations, number of submitted questionnaires, timely submission of questionnaires etc. have been implemented. For this scope, the monitoring indicators include: (i) Total user count (sharers, owners), (ii) Number of inactive users as inverse to usage of the vehicles and applications during a specific timeframe, (iii) Number of trips and questionnaires submitted per city on average on a timeframe. Other measures related to user activity monitoring constitute the reminders, e.g. mail or automated notifications via the apps, to users to take care of the questionnaires they are expected to fill in at different times during the whole demonstration phase.

4.5 The ELVITEN Dashboard

The usage of EL-Vs, trip data and questionnaires collection is bi-weekly monitored during the demonstration phase via a custom implemented interface, the “ELVITEN Dashboard”, a web application used for visualizing the data stored in the Data Warehouse through pertinent tables, charts and labels. It visualizes data from various sources including the trip data loggers on the EL-Vs, data from the ICT tools and the users’ responses to questionnaires and of the KPIs. The inspection of the Dashboard content sheds light also on reported problems and on the number of trips and questionnaires completed in each city (e.g. small number of trips or poor numbers in the submitted questionnaires can be easily spotted through the Dashboard, in order to timely address the cause of the problems and alarm partners to intensify their actions). Moreover, through the presentation of the historical and current views of operations, it can also serve as an alert mechanism by timely indicating possible integration problems between ELVITEN components. The various views of the Dashboard are the following (Fig. 2):

  • Cities Data Collection Overview: presenting statistics and counters for each city related with the number of registered users per city, trips, submitted questionnaires and problems encountered. Pie and bar charts have been incorporated in order to visualize the proportion of questionnaires and the proportion of ICT tools usage.

  • KPIs Overview: Visualization of the KPIs calculated at the beginning of each month.

  • Trip Data Tables View: exposes the content of the Data Warehouse related with the data coming from the trip data logger of the vehicles in the form of tables.

  • ICT Tools Data Tables View: exposes the content of the Data Warehouse related with the data coming from the ICT Tools and apps in the form of tables

  • Questionnaires Data View: exposes the content of the Data Warehouse related with the data coming from the questionnaires in the form of tables.

  • Dashboard SQL Explorer: A user-friendly interface in order to quickly write and share SQL queries through a simple and usable SQL editor.

Finally, the Cities Data Collection Overview is integrated to the ELVITEN website, namely “ELVITEN Dissemination Dashboard” to let users and other interested parties to be informed about the pilot activities and their progress (Fig. 3).

4.6 Issues Monitoring from Demonstrations

To ensure continuous and effective technical support to the demonstrations, the development of a dedicated issue-reporting functionality was implemented and included in the ELVITEN Unified application which integrates all the ELVITEN apps.

The issue-reporting functionality allows the ELVITEN users to report any kind of issues (i.e., concerning vehicles, infrastructures, or applications) that occur while using the ELVITEN services; in addition, users can report their suggestions about the ELVITEN services in general. However, major emergencies such as vehicles out-of-service and road accidents, can be reported to the local help desk by phone call directly via the Unifying App’s issue-reporting functionality. The Unifying App, by posing direct questions to the user, can evaluate whether the issue requires a direct phone call, and thus present the user with a phone call button or not. The GitLab [10] issue tracker has been chosen to send the issues on the basis of a decision tree.

5 Conclusions

This paper focused on the harmonized user-centered pilot methodology framework designed for setting the fundamental procedures which have been followed in the ELVITEN project, although its basic concepts can be used also in other technological domains, in order to involve and recruit users, get their consent, establishing the terms of their data processing and the agreements for the involvement of the EL-V owners and long-term EL-V sharers during the demonstrations, identify the current mobility situation of a pilot city and thus implement and deploy the necessary measurement, data collection and monitoring tools, so the targets set regarding creation of a big data bank can be met and consequently serve the analysis and evaluation procedures. Finally, it is worth to note that currently the data collected regarding the EL-Vs usage and users’ involvement, corresponding to about 34.000 trips, 400 vehicles in operation and 700 registered users, prove the efficiency of the ELVITEN demonstration methodology and will properly serve the analysis process, guidelines and recommendations extraction for planning authorities and manufacturers upon the end of the project.