Reproducing Paper Results

[st3lzer]

Hello,
I really appreciate your work and the extensive codebase!
I am trying to reproduce the results from the paper with the dataset from LMDrive.
For that, I fused both LiDAR folders and sampled at 2Hz (used every 5th image).

Anyway, I only get a DS of around 30-50 on the Town05 Long, which is caused by a quite low IS.
With the provided trained model to download from you, I achieve better results, so the error should not be in the benchmarking step.

This is my training script:

GPU_NUM=3
DATASET_ROOT='/beegfs/work/InterFuser/datasets/LMDrive/'
export CUDA_VISIBLE_DEVICES=0,1,3

./distributed_train.sh $GPU_NUM $DATASET_ROOT  --dataset carla --train-towns 1 2 3 6 7 10  --val-towns 4 \
    --train-weathers  0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 --val-weathers 0 8 20 \
    --model interfuser_baseline --sched cosine --epochs 35 --warmup-epochs 5 --lr 0.00025 --batch-size 85  -j 6 --no-prefetcher --eval-metric l1_error \
    --opt adamw --opt-eps 1e-8 --weight-decay 0.05  \
    --scale 0.9 1.1 --saver-decreasing --clip-grad 10 \
    --with-backbone-lr --backbone-lr 0.0001 \
    --multi-view --with-lidar --multi-view-input-size 3 128 128 \
    --experiment interfuser_baseline_2Hz_pretrained_rnd42_papervalues_trainseparated \
    --pretrained

Do you have any idea where this deviation in performance might come from?

Thank you!

[littleblackzi]

@st3lzer
Hello, I may not be able to answer your question, but I have noticed a new problem. In your training script, --multi-view and --with-lidar have no parameters. I noticed that the default parameter in the source code is False, as shown in the figure. If we need to train, do we need to pass the Boolean value True to --multi-view and --with-lidar?

[st3lzer]

Hello @littleblackzi ,
I think by parsing --multi-view as an argument, it gets automatically triggered to a true value. I tried it e.g. with the --pretrained flag.