Infrastructure-based automated valet parking is a growing business all over the world, with its origins in Germany. At the same time, it is one of the growing high-end technology solutions at Continental. This way of automating vehicle maneuvering has already been implemented in many locations internationally, the technology concept has been proven and the maturity is currently being developed.

In contrast to making the vehicle smarter by equipping it with more technology and “intelligence”, Automated Valet Parking type 2 (AVP2) is an intelligent infrastructure approach. The solution can be integrated into various kinds of infrastructure and has many potential use cases. Its original consideration is to relieve the driver of the unwelcome parking process at the end of a trip: When an AVP2 solution is installed in a parking garage, for example, at an airport or hotel, the system takes over vehicles at an entrance point to park them autonomously. Additionally, newly produced vehicles at a car maker plant can be driven remotely controlled by the AVP2 system to different stations in the factory. This is an economical method because it saves the typical waiting times of one-way-trips done by human drivers. Furthermore, automated vehicle driving via AVP2 potentially offers additional logistics use cases such as loading and unloading ships, or for loading cars onto a train plus many more applications.

Continental has introduced remote vehicle maneuvering within an AVP2 solution together with the car software start-up Kopernikus Automotive. This solution is based on a safe infrastructure, where the intelligence and processing power are installed into the building or site. To prove the concept, safe infrastructure solutions have already been deployed in garages and plants in Germany, in Asia and in the USA in several development projects.

The Infrastructure-based Autonomous Parking Procedure

AVP is low-speed automated driving (AD) for vehicles that do not possess the required sensors or intelligence inside the vehicle. Instead of using sensors and compute from the vehicle, they are installed in the infrastructure and only a drive command is sent to the vehicle. Vehicles are equipped with the technology to interpret and execute these drive commands, because the vehicle manufacturers design its vehicles with the automation use case in mind. Preparing a vehicle for factory automation means equipping it with an automatic transmission, electric power steering, electric parking brake, ESC, remote engine start/stop, and wireless communication. As a result, most new vehicles are ready for the public parking use case. The only thing that is additionally required is software that passes on the motion requests from the infrastructure to the vehicle actuators and ensures safety, for example, in case communication to the vehicle is lost.

The parking process works like this: The driver exits the vehicle at a designated area near the garage entrance and hands over the motion control of the car to the infrastructure. The most obvious tool for this would be a smartphone app. Now, the infrastructure and vehicle communicate to ensure that the vehicle is in fact eligible for AVP2 and is within the geofenced area for it. Once this is confirmed, a token is exchanged to complete the pairing-procedure between vehicle and infrastructure. The driver is free to pursue his or her activity at the destination and leave the rest to the safe infrastructure solution.

The car is now parked autonomously. It is guided by the control solution installed in the parking garage, Figure 1. Sensors in the building detect the car and perceive objects (Sense). Algorithms based on artificial intelligence (AI) interpret the sensor data - along with other tasks and technologies object detection and classification is performed - to create an environment model and calculate a safe path for each single vehicle in the garage on its way to an available parking spot or to the garage pick-up location (Plan). During this, the AVP2 system avoids any obstacles and finds a new path around obstacles. The required low-speed vehicle motion requests are communicated over the air to the vehicle. The motion software in the vehicle ensures that the motion requests are carried out by the actuators in the car (Act), Figure 2. When the driver has finished her or his activity, all she or he needs to do is to contact the infrastructure via the app and request the car to be autonomously driven to a designated pick-up area. The safe infrastructure starts the car and drives it remotely controlled to the exit point.

Figure 1
figure 1

An AVP2-ready car receives path information from the infrastructure (© Continental)

Full size image
Figure 2
figure 2

Principle split of the AD tasks within the safe infrastructure solution for AVP2 (© Continental)

Full size image

Functional Principle and System Components

AVP2 is based on sensors permanently installed in the infrastructure, AI- based algorithms that centrally move vehicles and traditional algorithms in the vehicle. Standard, low-cost surveillance cameras (IP-cameras) serve as main sensors, Figure 3. The camera signals are streamed to the control using the video compression standard H.264 (MPEG-4, part 10) for high-resolution applications. Complementary radar sensors optionally can be installed in areas with severe environmental conditions, such as very poor visibility. Data from both sensor types are fused and streamed to an industrial-grade server, installed in the building's climate-controlled server room. As the number of cars for AVP2 go up over time, additional servers can be installed to increase the system capacity. It is one of the concept's benefits that in contrast to in-car solutions, comparably strict size and weight limitations for ECUs do not apply to infrastructure.

Figure 3
figure 3

Standard surveillance cameras serve as main data source for vehicle localization and object perception (© Continental)

Full size image

AI-based algorithms on the AVP2 server interpret images from the video stream and thus localize the car of interest, detect any objects and object movement, that is vectors, in its environment, and then calculate a safe trajectory for the car. The reference grid for this is provided by a full digital scan and resulting digital map of the building. Camera-based localization places the vehicle on this map with an error margin of ±10 cm on a confidence level of > 99 %. Redundancy is provided by multiple cameras and sensors installed with a certain view field overlap in the building. Based on this, the path is calculated, and its information sent to the car wirelessly to allow steering to a parking space or to the exit.

The AI interprets what the sensors are picking up and discriminates between static and dynamic objects such as pedestrians, children, or bikes. When an unknown object in front of the vehicle is detected, the recognition system stops the vehicle and avoids contact via an emergency braking function. This has been demonstrated with volunteers briskly stepping out in front of an AVP2-controlled car, for instance during the IAA Mobility in Munich in 2021 and the CES 2022 in Las Vegas. This also works if the person approaches the autonomously driving car from the side.

The infrastructure-based sensor data processing for perception and path planning is a dynamic process calculating a path around static and dynamic objects. For instance, in the case of a dynamic obstacle, the remote control has the car wait until the route is clear again. If necessary, path planning switches to alternative waypoints. In other words, the AVP2 safe infrastructure solution is capable for mixed traffic situations. Figure 4 shows a small subset of the scope of potential situations reflected in the ISO 23374 features that need to be mastered.

Figure 4
figure 4

A small selection of the driving situations which need to be mastered by the AVP2 solution. (© ISO 23374-1 (Source: Intelligent transport systems - Automated Valet Parking Systems (AVPS) - Part 1: System framework, requirements for automated driving, and communication interface))

Full size image

Radio transmission between the infrastructure and the car can be done by a wide choice of wireless channels depending on the car maker's preference: Wi-Fi, 802.11p based vehicle- to-x (V2X) or LTE-based cellular V2X as well as 5G network slicing, 5G campus networks, and LTE. If the communication between the remote-control system and cars is lost for any reason, they will automatically stop on their own to assume a safe state. The car transmits wheel speed data, potential problems (such as a flat tire) as a status report and confirms actuator function to the remote control to close the feedback loop.

Current Development Status

Methods for certifying the AI-based safe infrastructure solution for AVP2 is being developed. As this is a new field, defining a process will take some time. One strategy for certifying the AI on a function level is to look at the same scenario in three different ways to detect if there is any dissent in the outcome. Compared to AD solutions for general roads, the situation is made easier with AVP2 as there is a more controlled environment (digitalized building and map of every static object/boundary), plus the driving speed is very low. Functional Safety of the AVP2 solution is mandatory. Continental brings its expert knowledge of, for example, IEC 61508/ISO 26262 to the development process.

Summary and Outlook

Low-speed remote maneuvering using standard IP-camera solutions and commercially available IT in combination with sophisticated AI algorithms enable a cost-optimized AVP2 solution. It is also a democratic approach to automation as one can control every vehicle, not only premium models with lots of cameras and computing power on-board. Continental has capabilities for many parts of the system, all the way from wheel speed sensors, to ECUs, to the infrastructure.

AVP2 technology moves fast, and pre- series systems have been installed in over 20 locations in Asia, in Europe and the USA. Currently the safe infrastructure solution is on the road to launch internationally. It is expected that this will lead to production ready AVP2 systems from 2025 and onwards. Developing AVP2 is motivated by the wish to make any operation involving the movement of cars on private grounds more efficient and cost effective. AVP2 offers the opportunity for vehicle manufacturers to automate the movement of vehicles in their production plants thus increasing the manufacturing and logistics efficiency. Drivers of these vehicles will ultimately benefit from a safe, comfortable, and time-saving parking process.