KING: Generating Safety-Critical Driving Scenarios for Robust Imitation via Kinematics Gradients

Project Page | Paper | Supplementary

This repository contains the code for the ECCV 2022 paper KING: Generating Safety-Critical Driving Scenarios for Robust Imitation via Kinematics Gradients. If you find this repository useful, please cite

@inproceedings{Hanselmann2022ECCV,
  author = {Hanselmann, Niklas and Renz, Katrin and Chitta, Kashyap and Bhattacharyya, Apratim and Geiger, Andreas},
  title = {KING: Generating Safety-Critical Driving Scenarios for Robust Imitation via Kinematics Gradients},
  booktitle = {European Conference on Computer Vision(ECCV)},
  year = {2022}
}

Contents

1. Setup
2. Scenario Generation
3. Fine-tuning
4. Town10 Intersections
5. Acknowledgements

Setup

Install anaconda

wget https://repo.anaconda.com/archive/Anaconda3-2022.05-Linux-x86_64.sh
bash Anaconda3-2022.05-Linux-x86_64.sh
source ~/.profile

Clone the repo and build the environment

git clone https://github.com/autonomousvision/king
cd king
conda env create -f environment.yml
conda activate king

Download and setup CARLA 0.9.10.1

chmod +x setup_carla.sh
./setup_carla.sh

Running the code

We provide bash scripts for the experiments for convenience. Please make sure the "CARLA_ROOT" ("./carla_server" by default) and "KING_ROOT" (if present) environment variables are set correctly in all of those scripts.

KING Scenario Generation

For all of the generation scripts, first spin up a carla server in a separate shell:

carla_server/CarlaUE4.sh --world-port=2000 -opengl

Then run the following script for AIM-BEV generation:

bash run_generation.sh

This script runs generation for all traffic density and automatically evaluates the results. For AIM-BEV generation using both gradient paths, run:

bash run_generation_both_paths.sh

Finally, to generate scenarios for TransFuser, first download the model weights:

mkdir driving_agents/king/transfuser/model_checkpoints/regular
cd driving_agents/king/transfuser/model_checkpoints/regular
wget https://s3.eu-central-1.amazonaws.com/avg-projects/transfuser/models.zip
unzip models.zip
rm -rf models.zip late_fusion geometric_fusion cilrs aim

Then change back to root directory of the repository and run:

bash run_generation_transfuser.sh

Fine-tuning

To fine-tune the original agent on KING scenarios, first download the regular data for AIM-BEV:

chmod +x download_regular_data.sh
./download_regular_data.sh

Then run the fine-tuning script (and adjust it for the correct dataset path, if necessary):

bash run_fine_tuning.sh

This script also automatically runs evaluation on KING scenarios.

Town10 Intersections

To evaluate the original checkpoint for AIM-BEV on the Town10 intersections benchmark, spin up a carla server and run

leaderboard/scripts/run_evaluation.sh

To evaluate a fine-tuned model, run the fine-tuning script above and toggle the commented "TEAM_CONFIG" variable in the evaluation script to change the model weights.

Acknowledgements

This implementation is based on code from several repositories. We sincerely thank the authors for their awesome work.

• CARLA Leaderboard
• Scenario Runner
• Learning by Cheating
• World on Rails

Also, check out the code for other recent work on CARLA from our group:

• Renz et al., PlanT: Explainable Planning Transformers via Object-Level Representations (CoRL 2022)
• Chitta et al., TransFuser: Imitation with Transformer-Based Sensor Fusion for Autonomous Driving (PAMI 2022)
• Chitta et al., NEAT: Neural Attention Fields for End-to-End Autonomous Driving (ICCV 2021)