Autoware Challenge 2023

[Planning] A planning architecture that achieves both high module-independent

scalability and feasibility of complex use cases
Challenge:

From the perspective of scalability and maintainability, it is desirable to develop functionalities such as intersection, pedestrian crosswalk, traffic light stop
planning, and obstacle avoidance, etc. with loose coupling. However, it has difficulty to achieve complex use cases where these functionalities are intricately
interconnected. On the other hand, in order to address such complex use cases, all-in-one planning algorithms have been proposed, but they have scalability
concerns in that each individual function cannot be independently extended.

This challenge is to propose the appropriate “software architecture and interface design” for the autonomous driving planning system that can address both of
these requirements.

Architecture with intricately interconnected functionalities: Architecture with independently designed functionalities: Examples of complicated use cases:
- Implementation of complex use cases: Easy - Implementation of complex use cases: Hard
- Scalability/maintenance: Hard - Scalability/maintenance: Easy oncoming car
Intersection module: Obstacle stop planner:
need to decelerate to avoid pedestrian on need to stop to avoid
oncoming car in front? crosswalk parked vehicles?
all-in-one
Planning Traffic jam
crosswalk
Avoidance planner:
avoidance need to steer to avoid
intersection avoidance crosswalk intersection parked vehicles?
Lane change module:
need to accelerate to avoid
obstacle stop oncoming car from behind?
traffic light
One module decision (e.g. lane change)
has mutual influence on the decisions of
lane change obstacle stop traffic light lane change Ego vehicle other modules and causes conflicts.
oncoming car
behind

Current planning architecture in autoware.universe:

https://autowarefoundation.github.io/autoware-documentation/main/design/autoware-architecture/planning/
[Planning] Natural driving planning algorithm for complex environments with high
density of dynamic objects
Challenge:

This challenge is to propose a “motion planning algorithm” that enables natural and safe driving even in crowded environments with high density of dynamic
objects. It is desirable that the algorithm have the following practical features as an autonomous driving system: "to adjust behavior according to object types,
similar to how a human driver reacts differently to a car versus a motorcycle", and "to account for recognition errors".

Example Scene 1: Example Scene 2:

Low-speed zone with mixed traffic including pedestrians, High-speed zone with merging required using rapid acceleration or deceleration. Braking
bicycles, and vehicles, with unclear lane boundaries. may lead to dangerous situations if there are vehicles approaching from behind the
current lane.

https://www.youtube.com/watch?v=KnPiP9PkLAs
https://www.youtube.com/watch?v=RRfbrdXmObg
[Planning] Planning system capable of handling self-position information from relative
and absolute coordinate systems
Challenge:

In autonomous driving systems, the self-position is divided into two types: local coordinate and global coordinate. When the global coordinate is available,
many information can be obtained through high-precision map information, for example, the position of traffic lights. However, calculating the global
self-position can be sometimes difficult in places such as in tunnels in high-speed highway. Moreover, even in urban driving, there are also places where a
global localization is difficult. In such cases, planning and control needs to be performed in the local coordinate using a relative position in the driving lane
obtained by white line recognition, which can give limited information compared to the global localization. When the characteristics of localization change
drastically, handling them seamlessly is not easy, especially for complex planning modules.

This challenge is to propose a planning system that properly handles switching between local and global coordinate systems when self-position switching
occurs.

Expected use case 1: Expected use case 2:

Approaching Highways from Global localization is temporarily
Urban Areas unavailable in urban area
Local Coordinate
Localization Local coordinate only
100
Planning system

Global Coordinate
Localization

60
[Planning] Evaluation metrics for planning in autonomous driving

Challenge:

Ensuring safe and appropriate driving in autonomous vehicles requires more than just following traffic rules. It is also important to consider
whether the vehicle's actions are dangerous or not, for example, blind spots, pedestrian sidewalk, interactions with other vehicles. The
definition of such evaluation metrics is essential for the evaluation of automated driving.
However, list them in all situations considering traffic rules causes a very large number of metrics. “How closely autonomous driving resembles
human driving” is one of the metrics, but human driver’s behavior sometimes includes an unsafe behavior in mathematical point of view.

This challenge is to propose potential evaluation metrics for identifying "necessary and sufficient" indicators of good autonomous driving.

Example 1 Example 2 Example 3 Example 4

Appropriate distance from Presence of dangerous Absence of unnecessary Smooth driving trajectories
preceding vehicles maneuvers acceleration and deceleration
[Control] Algorithm for achieving high-precision vehicle control for vehicles with various
charactaristics
Challenge:

Vehicles equipped with autonomous driving software have various motion characteristics depending on the vehicle type. For example, the Drive-by-Wire
(DBW) performance of vehicles varies according to vehicle types such as passenger cars, small cars, large special vehicles, buses, etc. Additionally, in
practice, there are many complex elements that make modeling difficult, such as differences in brake response speed, delay in instruction, existence of
steering dead zones, insufficient control cycle, and unpredictable behavior due to the intervention of black-box software. To make autonomous driving more
widespread, an algorithm that can be quickly applied to such a wide range of vehicles is needed.

This challenge is to propose an algorithm that can achieve optimal control based on the given vehicle's driving data. It is assumed that the vehicle has an
ackermann type of steering control mechanism.

It is also desirable to be able to demonstrate the limit of the control performance. For example, if the assumed vehicle performance from this data results in a
50cm tracking error even with the most optimal control. The method of demonstration (theoretical proof / numerical simulation, etc.) is not specified.

various vehicle charactaristics

Autoware

target steering/accel/brake
Analyzing driving data:
- propose suitable control
Vehicle
logic/parameters.
DBW system - propose performance limitations.

vehicle actuators
[Sensing/Perception] Improvement on object recognition in both accuracy and stability

Challenge:

This challenge is to propose online perception algorithms that have accurate/stable detection results as we see in Waymo videos, to solve below issues

- Accuracy of detected position/size/orientation of parked vehicles and large vehicles in close range
- Mis-interpretation of poles or traffic signs as pedestrian
- Detection accuracy of vehicles in long range with occlusions
- Object recognition result is unstable for objects when it is temporarily occluded

Blue: DNN based detection

Green: Tracking

https://www.youtube.com/watch?v=KnPiP9PkLAs
https://www.youtube.com/watch?v=RRfbrdXmObg

Example Scene 1: Waymo perception from videos

Autoware perception result: incorrect vehicle orientation, multiple results being - https://youtu.be/b84FbSZFJeQ?t=120
output from one vehicle, insufficient stability over time. - https://youtu.be/ospoTAyEdDQ?t=74
[Sensing/Perception] Improvement on robustness of object recognition

Challenge:

This challenge is to propose online detection algorithms/models that are robust against different environment and different sensors.

Current our detection model’s performance deteriorates when:

- driving in different environment where training datasets are collected (e.g., driving Japanese roads using models trained by open datasets)
- Changing sensor configuration (using different lidar model, changing number of lidars)

Some possible approach might include:

- Establish domain adaptation method
- Designing robust DNN model
- Generate enough dataset that includes various sensor configuration and various driving environment

Green: detection result trained by real vehicle dataset

Yellow: detection result trained by open dataset

Example Scene 1: Example Scene 2:

The model trained on the open dataset recognizes trailers in the image as two Detection of pedestrians carrying an umbrella is unstable (recognized as a cyclist, but
separate vehicles, but the model trained on real vehicle data can correctly sometimes cannot be recognized at all).
recognize them as a single vehicle.
[Sensing/Perception] Detection of blinkers of surrounding vehicle

Challenge:

Current Autoware does not have a module that can detect blinkers or lamps of vehicles. It only uses vehicle’s current motion to predict its future path. Due to
latency in detection and tracking, we have large accumulated latency in the prediction result.

This challenge is to propose a detection module to detect the state of blinkers/lamps of surrounding vehicles.

This would allow us to recognize the vehicle states to anticipate its future behavior. It would be nice if the module satisfies the following condition:
- Capable to operate in realtime (10FPS) on an edge device (e.g., Jetson AGX Xavier)
- If the module requires too much computation to detect all the vehicles’ blinkers, it is also expected to select appropriate vehicles to detect their blinkers

https://www.youtube.com/watch?v=KnPiP9PkLAs

Source: AC photo Source: AC photo

[Sensing/Perception] LiDAR filtering false detection suppression

Challenge:

When performing obstacle detection using rule-based LiDAR point cloud filtering, the task of this function is to remove various noise point
clouds from the entire point cloud. To avoid sudden braking, it is necessary to remove noise points, but it is also important to keep the points of
obstacles that pose a collision risk.

The following challenges arise:

- Unfiltered point clouds of rain, fog, dust, birds, and other objects can remain.
- It is difficult to distinguish between LiDAR reflection points that are not a collision risk (such as plastic bags, water vapor, exhaust gas, noise,
and ghost caused by multiple reflections of LiDAR, etc.) and those that pose a collision risk. This can lead to false braking.

Example Scene 1:

The exhaust gas emitted from the vehicle has been detected
as a point cloud in the air.
[Localization] Improving Localization in Dynamic Environments with Inconsistent Map

Challenge:

Objective:
There are cases where the map data does not match the real-world environment, such as in parking lots or factories that change over time. Although PCD and
LL2 data are provided, they may not be reliable. We want to ensure consistency with the existing map while estimating the position in the missing areas using
some method.

Assumption:
PCD and LL2 data are available for the planner.

This challenge is to propose a localization system properly estimating the position in the missing areas.

Ex. Inconsistent Map:

Dynamic occlusions Major changes in structure
Lack of PCDs or lines in map

NO
PCD
map

LiDAR
measurement
Source: TIER IV Source: free-materials.com Source: Google Map The parkhouse Akabane front Aug. 2022
[Localization] Improvements in initialization of localization algorithm

Challenge:

Objective:
Autoware cannot initialize its pose in environments without GNSS signal. Even with GNSS, there is also issues occasionally falling into wrong pose. We would
like to have an algorithm to estimate initial pose globally within a given map.

This challenge is to propose a localization initializing the position globally without GNSS and initial values.

Ex. Sample solutions for similar issue:

hdl_global_localization

Wrong
Initialization

Source: TIER IV
[Localization] Estimation of errors in the environment hard for localization

Challenge:

Background: Autoware currently use point cloud matching method to localize its pose. However, It can’t detect the suspicious state in the situation where it
can be unreliable like the following:
Sample Scenario 1: Vehicle is driving in a tunnel. It might be accurate in lateral direction but not in longitudinal direction.
Sample Scenario 2: Vehicle is driving in an open space. It might be accurate in z/roll/pitch, but not in x/y/yaw

This challenge is to propose an algorithm to detect the unreliability of localization in realtime.

Sample scenario 1: Related Research

tunnel

to estimate the unreliability that says "it's hard to

matching this direction." Source: TIER IV