Virtual Modeling of an Electro-mechanical Powertrain and Steering System with Optical Proximity Sensors for Driverless Ambulance Vehicles in Unity 5

Abstract

Since automated driving and vehicles are gaining popularity, the optical parking systems, electric power steering (EPS) solutions, as well as the electronically controlled powertrain, represent important technological landmarks. They facilitate driving now by improving the ergonomic features and through their specific function of lowering the physical stress and specific efforts that are required from the driver or user in the real-road operating conditions, such as steering, parking, cornering, forward and reverse maneuvers. Also, electric powertrain and steering systems give the possibility and all basic components for high precision and advanced control of the most important systems (such as steering, braking, propulsion, and suspension), thus improving the infrastructure and making the way to autonomous operation and automated driving capabilities. In these conditions, the present scientific paper shows a virtual modelling process of an electric powertrain and steering system designed with optical proximity sensors for a battery-electric four-wheel-drive smart prototype ambulance as important component in Emergency Medical Unit (EMU) fleets. Specifically, the vehicle control system is working based on a linear actuation model of the steering and acceleration. The ambulance is tested in virtual reality with Unity 5 application. Modeling and testing techniques for this automated ambulance are outlined. Thus, virtual model is tuned through the training sessions. Finally, the performances of the EMU vehicle are tested virtually on the Unity platform with artificial intelligence. The powertrain and propulsion systems performance (especially steering, but also braking and suspension) are measured in terms of the percentage or capability for following the reference track, thus showing important outcomes in the case of the proposed architecture.

1 Introduction

Driverless vehicles seem to be both a necessity and utility in the future of transport and for a segment of the road traffic. It is an both a practical idea and an ideal scenario to use as much as it is possible driverless robotic vehicles for specific tasks such as highly predictable traffic and transport tasks. There are multiple companies and developers of automated vehicles that compete in the field. One such company is Alphabet Inc which has indicated the necessity for autonomous or automated driving technologies since 2009. This company came into the playground of automated driving industry with the Google vehicle, known as Self Driving Car Project, and today has its own independent automated vehicle (AV) tech project, popularly indicated as Waymo.

Another player, such as Intel Corporation’s project known as Mobileye, made an important step in taking up the challenge and task of making complex experimental tests with the AV in the City of New York. The semiconductor companies are rallying for the AV development and testing. Luminar company develops the specialized lidar technology for automated vehicles, such as passenger cars and cargo trucks. Practically, this company, designs, develops, and finally sells multiple long-range lidar components (Hydra and Iris sensors) and systems for AV. Ambarella, the popular computer vision chip designer, is a different important player in the AV technology. It makes the design and development for high definition (HD), ultra-HD video codding/compression, specific processing of images, and digital solutions for computer vision, offering important support in blind-spots visual detection. It offers components, assemblies and products that are frequently applied in multiple computer vision applications. Latter one includes automated vehicles with autonomous driving capabilities, reconning vehicles, advanced driver assistance systems (ADAS), and specialized robots. In the first part of the year 2021 CV2FS CVflow AI vision processor has been created to be installed on the Arrival Buses and Vans due to its capability to enhance ADAS features, and autonomous driving. These systems provide the necessary artificial intelligence based on neural network processing power, environmental perception, stereovision capability, and HD image acquisition. Aptiv PLC designs and develops electric, and electronic components for active safety systems in automotive industry. The company has recently made a project with Hyundai, known as Motional, to test the AVs’ capabilities for monitoring the infrastructure in real-time.

Qualcomm Inc created a platform project, known as Snapdragon Ride, that allows the automotive designers and producers to custom program their AVs for specific tasks and requirements through a programmable framework. MicroVision company has developed a scanning technology with laser beams, light detection and ranging (LiDAR) to create high precision maps of AV surroundings and objects. The minimal elevations of the detected objects and obstacles is at an accuracy around 10 cm. The onboard digital control units use maps of the surrounding environment to navigate and drive the AVs.

General Motors, an important and well-known automaker, supports the AV technologies, through Cruise, a subsidiary company. It is testing AVs for passenger rides in California, with the support from other companies like Microsoft and Walmart. Tesla, NVIDIA and XPeng Motors are also very important players in the field of AVs design and development. Designing, researching, and developing program models for controlling with artificial intelligence the AVs both in virtual reality (VR) and in actual road traffic is still a big challenge, because some of the over-the-air updates may pose as security weaknesses in some cases [1, 2]. Defining the system-on-a-chip electronic control units, the interfaces and the programs used for the mobile platforms and automotive industry, facilitates the digitization and computing process in AV technology [3]. Artificial intelligence applications are to be used both in electric vehicles (EV) and AVs, to improve safety and accessibility [4,5,6,7].

The new crowdsourcing program for micro-mobility drive-sharing platforms outlines the challenges and weak spots of the AVs in road traffic and transportation system, operated and exploited based on integrated demand-supply relations. Different scenarios are simulated to gain the proper data [8]. Energetic efficiency and operation safety may be improved through the implementation of the artificial intelligence [9]. To avoid accidents, road traffic events, both AVs, and connected cars, must scan the surrounding environment to properly detect the other road traffic participants and the objects in their proximity [10]. In hazard scenarios, data transmission between different vehicles and road users faces multiple challenges such as the message flooding or contention, and broadcast storm [11].

The Fig. 1 presents some of the scenarios in which the communicating vehicles may reside during the road use and when communicating to each other. The [11] research work showed us that it is possible to create a topological sequence recognition mechanism to generate a connected car‐based subgroup. This leads to a recognizable geolocation definition in real time. It facilitates the position characterization in relation to the dynamic topological variation of the vehicles in the subgroup. They applied this mechanism in the Connected Mobile Virtual Fence (CMVF) and have tested its capabilities in real traffic. To avoid road traffic events, both AVs and cars with advanced driving assistant system (ADAS) will be considered safe when relevant data are available (obtained, process and eventually displayed) about vehicles in proximity [11, 12].

Through the appearance of vehicular ad hoc networks (for short known as VANETs) and the so-called Internet of vehicles (IoVs), there is produced a large amount of digital data. These are available both for vehicle users and traffic control systems. The amount of available data is proportionally related to the density of road traffic and participants. It is higher than vehicle’s processing and storage capacity. To handle this challenge, VANET is supported by combined cloud computing systems. The later one is also known as vehicular cloud computing (VCC). It manages vehicle’s data and offers support to the drivers [13]. Proposing a secure key agreement with authentication protocol to obtain a data message confirmation through VCC the [13] protocol is proved as safe during attacks. Meanwhile it offers privacy to the users and mutual authentication.

Fig. 1.

Basic scenario models for interacting vehicles in road traffic communications system to each other [11]: (a) independent vehicles in road traffic; (b) forming a group and overtaking step; (c) group initiation and overpassing; (d) reformation of the group and overpassing; (e) full connected group and its topologic phase; (f) complex all-to-all overpassing.

In the Fig. 2 is shown the communication protocol between the vehicle and the infrastructure, as well as the internal data transfer for autonomous driving and steering, supported by [13, 14]. Vehicular cloud 1 allows information exchange 2 with the server of the infrastructure 3. Mutual authentication and session key agreement 4, as well as registration 5 of the vehicle to the server 3.

Fig. 2.

Schematic of the communication model vehicle to infrastructure and onboard data management: (a) external communication [13]; (b) internal data transfer [14].

Vehicle 6 sends and receives packets of data for message confirmation 7, mutual authentication and session key agreement 8 to the roadside units (RSU) 9. Like these transfers is data exchange of infrastructure components for registration 10, mutual authentication and session key agreement 11. Information exchange 12 takes place between vehicular cloud 1 and the road infrastructure 9. Considering the technical aspects, the challenges with detecting obstacles at higher speeds and long distances may be the greatest difficulty to solve. In relation to road traffic control, most studies support the idea that vehicles should exchange information. General vehicle to vehicle (V2V), vehicle to infrastructure (V2I), vehicle to everything (V2X) cooperation, communication, vehicular ad-hoc networks, and platooning are solutions being studied, with multiple variables [15]. Experimental research of the management system and data exchange should allow the optimization of the electronic control for the vehicles [16]. Integration of the mechanical and electronic components in the design and modelling of mobile robotic applications [17], as well as AVs, is the basis for development of smart cars. Virtual modeling and testing in real road traffic conditions are the way to develop and design electric and smart vehicles [18]. Simulation approach as vehicle-in-the-Loop (VIL) is an innovative way for analyzing ADAS vehicles that is the basis for the safety in the critical testing scenarios. In real-road traffic tests some variables are specific but some of them may be recreated based on virtual environments [19]. By development of the automated systems the mobile tracking, vehicle location and remote accessibility are closer than they appear to be [20]. Lidar technology and cameras installed on the vehicles are two solutions that are considered in the automated driving [21]. Light detection and ranging (LiDAR) system improves the features of monitoring the surrounding traffic and supports the advanced driver assistance systems. With the 360-degrees capability of monitoring the proximity it should improve the quality of safety alerts and their accuracy also. Some of its critics support the idea that Lidar is inferior to the cameras, as well as more expensive.

In Fig. 3 is presented the compared analyze for both systems of proximity monitoring.

Fig. 3.

Simplified models of proximity safety control: (a) camera-based system with visual image analyze; (b) LiDAR sensor range with 360-degree vision that solves the problem of blind spots.

Virtual scenarios simulations are the proper method for studying the context and performance in given conditions for the considered system or vehicle. Autonomous driving simulators allow the researchers to recreate some of the road traffic and transport conditions to find solutions and solve the problems [22]. The main objective of the present paper is to show the steps and challenges with the virtual modelling of an electric powertrain and steering system for an automated ambulance in virtual simulation and with ADAS in real road traffic. The virtual model of control unit has been designed, studied, and tested in Unity 5 for an automated ambulance. Other objectives are: using the optical proximity sensors; making the SWOT analyze and interpretation of the results.

2 Materials and Methods

The method applied in this study is mainly based upon virtual modeling, simulation, and validation experimental tests. In the first step, there is created a virtual model and after that the practical set up validates the simulation results on a real size model. In the last two years, with Covid pandemic, the robotization, automation, and digital modeling were speed up. Anyway, the obtained results did not satisfy all the proposed expectations. Communications were certainly gaining profit, but multiple other fields have not recorded significant benefits. Technology advancement toward robotization and automation are missing still from the field of emergency rooms and ambulances. Increasing digitization process with “online school” and other “online procedures” in public institutions and companies made an upgrade in software, hardware, and methods of interaction. Although the things were progressing in some parts, other systems and their related activities are less suited for upgrading with highly automatic, robotic, and digital equipment. The emergency rooms and healthcare system in general has been strongly stressed and used during the COVID-19 pandemic crisis, but for example it was not profiting fully from the existence of the AVs. Ambulances, during COVID-19 crisis, have been stressed and challenged by emergency events and hazardous scenarios which haven’t been managed always as expected. The ambulances and emergency personnel must be very responsive and act rapidly to save lives and protect the health of individuals in need of help. Those employed in this activity are frequently stressed out, exhausted and overworked. The existence and use of AVs as robotic ambulances to transport some patients and health care seekers who are searching for medical attention could be practical and a life-protecting measure. The present paper searches all the materials and methods that may be applied and tested on an AV model suitable for ambulance services. Using the modern technologies to optimize the emergency services with AVs, robotic devices, and digital procedures may bring benefits both to the staff and society. The alternative solutions and procedures may be applied to increase life quality, to mitigate stresses in health risk situations and to provide better services. If there is no need for complex resuscitation procedures on the patient the AV ambulance seems to be a proper solution. If there is required transportation between home and emergency unit, or between different health care units, the use of AVs would be the proper solution to ease up the process. These additional solutions are somehow adequate and realistic now because they lead to an improvement of emergency units. The powertrain is one of the few most important systems of a mobile robot or AV. Its operation must be either defined through calculation or measured experimentally and practically to be considered when starting the main stages of the AV virtual modeling and programming process. This is so because of the wide variation in the kinematic and dynamic parameters defining the operation. The appropriate methodology for the research, design, and development of a robotic ambulance (as AV) is the virtual modeling and simulation, followed by the experimental test and optimization according to the real-world scenarios. Unity 5 is the proper application to be used for creating the virtual model and to simulate the AV. Different scenarios are studied to gain the realistic data relative to the kinematics and dynamics of AV in virtual and actual conditions.

Equipment used in the research is the Unity 5 software and a real vehicle platform with ADAS for testing and inspecting capabilities of the optical proximity sensors, if suited for automated driving. Technical data brief overview is shown in Table 1.

Table 1. Technical specifications regarding the AVs used for the present study.

3 Virtual Modeling and Results

Results of the virtual modeling both in CAD apps and in Unity 5 environment are compared with the actual data. Important findings of the virtual modeling and research through simulation, followed by testing, outline some kinematic and dynamic limitations for the studied AV model. Safety operation has been reached at speed ranging between 20–45 km/h (5.5–12.5 m/s). Data available regarding tires status, road surface friction coefficient variation in the virtual modeling is still needed to be defined in higher level of detail.

Fig. 4.

Simplified CAD models of vehicles with electro-mechanical powertrain and OPS for study: (a) virtual model used in simulation; 1-wheels, 2-batery, 3-electric connections, 4-electric engine, 5-battery control unit, 6-front OPS hub, 7-engine control unit, 8-antilock braking system unit, 9-onboard central processing unit, 10-steering unit, 11-mechanical transmission, 12-rear OPS hub, 13-OPS electrical connectors; (b) schematic of the physical powertrain with OPS capabilities; 1-wheels, 2-engine, 3-clutch, 4-gerbox, 5-engine control unit, 6-front OPS hub, 7-transmission control unit, 8-antilock braking system unit, 9-onboard central processing unit, 10-steering unit, 11-rear axle, 12-rear OPS hub, 13-OPS electrical connectors.

Unity 5 environment, known as Virtual Reality platform, offers the programing possibility with artificial intelligence to control the AV model. Figure 4 shows the simplified schematics of CAD models for the vehicles equipped with OPS and used in the present research.

Robotic ambulance virtual model in Unity 5 has advanced automated driving capabilities supported by AI. Selecting and defining the equipment and materials in the Unity program is shown in Fig. 5.

Fig. 5.

Unity 5 panels for scene collection and material definition regarding the virtual model of the AV: (a) equipment and scene collection menu; (b) materials definition board in VR application.

The actual car model than facilitates the validation of the program, with its advanced driver assistance system (ADAS). In the first place, it is quite important to consider the fact that, nowadays, autonomous, AVs, or self-driven cars do not appear for sale in the market to everyday customers. Platforms like those popular cars made at Tesla factory, or the Super Cruise-equipped Cadillacs are featuring the capability to ride and use the hands-free technology for limited (but significant) time intervals. They do this for a precise and limited set of conditions (meaning highway and interstates).

Modeling the car powertrain, OPS and surrounding environment is performed with in Unity 5, by applying those scripts and menu boxes for data definition, according to Fig. 6.

Fig. 6.

Basic software features for defining the surfaces with Blender script for Unity 5 program: (a) Surface virtual modeling editor; (b) Setting menu for Blender application in VR environment.

To model the virtual driverless ambulance (as an AV) in Unity 5 is necessary to apply the general and specific editing features, shown in Fig. 7.

Fig. 7.

Editing features for compositor, shader, images, and animation in Unity 5 digital platform: (a) General features in Editor type section; (b) Animation features menu useful to run the VR model.

Other specific editing features regarding scripting and data, are shown in Fig. 8.

Fig. 8.

Editor type in Unity 5 application for scripting and data: (a) Scripting for the following options: Text Editor; Python Console; Other Info; (b) Data menu for the following: Preferences, File Browser; Properties; Outliner.

Results obtained through virtual modeling are based on artificial intelligence protocol which is using vehicle-to-everything and context awareness data systems. Automated robotic vehicle is thus generated and tested in Unity 5. Object perception and situation awareness capability are important requirements to perform avoidance maneuvers around the obstacles. The AV with OPS is designed to drive around the objects for completing the transport task. Dynamic parameters such as the following: resistance air force (Raf), gravitational force (Fg), road vertical reactions to weight force on each axle of the vehicle (front axle reaction = Rf, rear axle reaction = Rr) and vehicle speed (as kinematic parameter) are most important to be considered in virtual modeling, along with engine’s output performances (such as torque and power). In Fig. 9 are presented the AV’s resulted forces.

Fig. 9.

AV robotic ambulance model in Unity 5, used as Virtual Reality drive-test program: (a) dynamic parameters definition as forces; (b) Torque and longitudinal velocity in object’s proximity.

The developed tests are consisting in straight road segment and follow the track, sine-sweep steering maneuver, and steering maneuvers created by the AI program which controls the driverless AV dynamics. The performed sine-sweep procedures are followed with both virtual model and physical vehicle. It aims to the system characterization and dynamic behavior definition. Anyway, the AV’s tracking performance is evaluated when the reference road profile generated by the program for the autonomous task is also offered. It is important to consider the integration of new technologies, such as Connected Mobile Virtual Fence. If AVs are equipped with AEB (Autonomous Emergency Braking system), the vehicle will almost instantly and automatically operate the emergency brakes. This is done when road traffic is moving around the AV, when they overtake and intervene in car’s trajectory. Virtual Fence Technology makes the idea of continuous communication possible to avoid events and accidents. It is the basis for mutual communication of AVs to increase their awareness or perception factor. Vehicle-to-vehicle (V2V), vehicle-to-infrastructure (V2I), and vehicle-to-everything (V2X) protocols must be implemented to provide the connected car feature. When AV do participate in the road traffic, the volume of information exchange needed and the frequency at which it must be acquired and processed is tremendous. To have a decision-making system similar in complexity with a human brain, the AV must be provided with a real-time image of the context around them. This is quite accurate in road traffic urban environments, where human drivers encounter and perceive in road traffic other individuals, different animals, as well as a multitude of vehicles in a small timeframe. Vehicle-to-vehicle communication is the basis for the creating the mutual awareness system. The performance of the studied AV with electromechanical powertrain and steering system was investigated based on the driving profile based on an autonomous tracking mission. The AV is travelling at a predefined longitudinal speed of 10 m/s while the steering wheel rotation is modified by the electronically controlled actuator (2) for a time of 60 s (see Fig. 2b). In automated driving scenario the steering angle value is determined through the encoder sensor (10) provided in the virtual model (see Fig. 4). The present research is based on multiple protocols which allow the optimization of the Connected Mobile Virtual Fence (CMVF) system using LiDAR, smartphones, cameras and/or the connected vehicle technology. A connected car is the one that is using wireless networks to communicate with nearby devices. Connected cars are a big step forward toward the IoT. The AI program distinguishes the direction to which vehicles are traveling and allows individual cars from the connected group to communicate and mutually recognize each other’s position. It gives a precise topological sequence. That is one very important task, and when the preceding car gets closer to an event, the accident may be avoided by warning the following traffic in advance about the hazardous situation of the preceding vehicles. As vehicles travel in the other direction, they don’t need to read data of those driving forward, thus the risk of message contention or congestion to send and receive false emergency encoded messages may appear rarely. Unity 5 is useful both for environment development and for testing whole vehicle as shown in Fig. 10.

Fig. 10.

Virtual model of a driverless ambulance vehicle in Unity 5: (a) driverless ambulance; (b) Connected Mobile Virtual Fence and tracking in Unity 5 software application.

The “Fitness” is determined based on the distance traveled by the vehicle, the average speed at which the car travels, as well as the “Sensor Multiplier” value that considers how close the vehicle is to the road’s margins during driving. The distance traveled by the car is in fact also the decision-making component in the calculation of “fitness”. This component of calculus has an important role in determining the “fitness” value. Other factors are also important, namely” Avg Speed Multiplier” and “Sensor Multiplier”, as shown in Fig. 11. They are referring to the average speed of the vehicle and the position of the vehicle on the road. They are considered, but the role played is less important compared to the distance traveled by the car. When lacking one of these components which allows the assessment of how the vehicle travels the route, it is quite hard to arrange them in such manner that the AI of AVs may learn. Context Awareness Mechanism based upon Mobile Virtual Fence System helps the artificial intelligence program to control driving actions, with secure maneuvers and direction change, to avoid the physical objects.

Fig. 11.

Unity code script: (a) The code which makes possible the calculation of the “Fitness”; (b) “Fitness” component code presentation.

Using the Mobile Virtual Fence System “(MVFS) in simulation and in real vehicle (with ADAS) is very important for automated driving process optimization. MVFS is generated via OPS. Other technologies may be also used (LiDAR, smartphones, cameras etc.). Each obstacle is triggering the OPS and subsequently leads to signals and actions related to context aware mechanism. To avoid accidents, AVs or conventional vehicles equipped with advanced driving assistant systems can be labeled as safe if specific data can be obtained on the scenario of other object in the proximity. Real world traffic is the ultimate basis for the detection of proximity obstacles and for vehicle mutual awareness in Connected Mobile Networks through contactless and wireless information exchange.

With Context Awareness Mechanism (CAM), supported by OPS, the real vehicle in obstacle approaching situation is signaling and acts (visually, acoustically and emergency braking if needed) if the probability of an accident is increasing, as shown in Fig. 12.

Fig. 12.

Real vehicle is equipped with OPS both in front and rear side to detect physical objects and avoid events: (a) with ADAS field of awareness is 2 m long and 2 m wide; (b) Different states of contactless perception of the obstacles in the front of the vehicle.

Virtual modeling of an electro-mechanical powertrain and steering system with optical proximity sensors for a driverless AV, designed to operate as an ambulance, with the artificial intelligence support, both in the Virtual Reality and for real road traffic, encounters problems and challenges, which are presented in Table 2. OPS support the design and development of partial or complete virtual fence. The later one signals the presence of objects in the proximity of the vehicle. Wireless capabilities and/or hot spot option on smartphones facilitates the implementation of a mobile virtual fence.

This one may be used to exchange information with the surrounding environment. Using the mobile virtual fence (MVF) through smartphones one may almost directly join or initiate a mobile vehicular network through which the important traffic data may be sent and received. Context aware mechanism based instead on OPS, is made by the proximity sensors and the electronic processing unit which calculates the data received simultaneously from all channels or sides of the car, is controlling behavior of the vehicle in hazardous conditions. Evaluation and centralizing of the strengths, weaknesses, opportunities, and threats highlights a package of the most important observations and hotspots, as shown in Table 2.

Table 2. Observations of AVs’ Strengths, Weaknesses, Opportunities, and Threats (SWOT) research.

Writing the AI program based on Bayes Theorems is using the probability factors. The first or primary probability is considered by the AI and defined using a training group as it is shown below:

$$ {\text{Q}}_{\text{I}} \left( {{\text{x}}|{\text{w}}} \right) = \left[ {{\text{Q}}_{\text{I}} \left( {{\text{d}}|{\text{w}}} \right) \cdot {\text{Q}}_{\text{I}} \left( {\text{x}} \right)} \right]/{\text{Q}}_{\text{I}} \left( {\text{w}} \right) = \left[ {{\text{Q}}_{\text{I}} \left( {\text{x}} \right) \cdot \Pi_{{\text{i}} - 1}^{\text{N}} \cdot {\text{Q}}_{\text{I}} \left( {{\text{w}}_{\text{i}} |{\text{x}}} \right)} \right]/{\text{Q}}_{\text{I}} \left( {\text{w}} \right), $$

(1)

where Q_I(x) is the first probability; Q_I(w) – final state probability; Q_I(w|x) – probability of training group.

4 Conclusions

Designing, virtual modeling, testing, and experimenting with digital control programs in Unity 5 for AVs and ADAS equipped cars has been a step forward and supported us to understand the challenges which are important to be studied in further research, both through simulations and road traffic measurements. Testing in road like scenarios on actual vehicles is still a challenge because there are no series produced AVs yet for daily customers. There are only concept cars and limited series models with different levels of automation. The conventional advanced driver assistance systems, on the other hand, are quite useful for the vehicle’s kinematic and dynamic control in hazardous situations, but still needs the presence and actions of the human driver.

Some of the significant and challenging topics that were studied in the present work are as follows: virtual modeling of an electro-mechanical powertrain; designing the power steering system on the same virtual model; indicating the architecture of the optical proximity sensors installed on the passenger car with ADAS. Primary phase of research consists in simulations with Unity 5 program and then comes the second step which makes practical measurements of distances between the vehicle and the surrounding objects with ADAS and OPS in real road traffic. To avoid traffic events and crashes, ADAS that are nowadays installed on conventional cars are a step forward but neither perfect systems nor complete for the automated driving scenarios. They are lacking both the feature of connected car technology and self‐driving capabilities.