Skip to content

Perception Evaluation#

Preparation#

  1. Copy sample scenario

    mkdir -p ~/driving_log_replayer_data/perception/sample
cp -r ~/autoware/src/simulator/driving_log_replayer/sample/perception/scenario.yaml ~/driving_log_replayer_data/perception/sample

  2. Copy bag file from dataset

    mkdir -p ~/driving_log_replayer_data/perception/sample/t4_dataset
cp -r ~/driving_log_replayer_data/sample_dataset ~/driving_log_replayer_data/perception/sample/t4_dataset

  3. Transform machine learning trained models

    source ~/autoware/install/setup
ros2 launch autoware_launch logging_simulator.launch.xml map_path:=$HOME/autoware_map/sample-map-planning vehicle_model:=sample_vehicle sensor_model:=sample_sensor_kit
# Wait until the following file is created in ~/autoware/install/lidar_centerpoint/share/lidar_centerpoint/data
# - pts_backbone_neck_head_centerpoint_tiny.engine
# - pts_voxel_encoder_centerpoint_tiny.engine
# When the file is output, press Ctrl+C to stop launch.

How to run#

  1. Run the simulation

    dlr simulation run -p perception  -l play_rate:=0.5

    perception

  2. Check the results

    Results are displayed in the terminal like below. The number of tests will vary slightly depending on PC performance and CPU load conditions, so slight differences are not a problem.

    scenario: sample_dataset
--------------------------------------------------
TestResult: Failed
Passed: criteria0 (Success): 215 / 215 -> 100.00%, Failed: criteria1 (Fail): 0 / 8 -> 0.00%