Strange Evaluatoin Result on Kitti

[aliko70]

Hi @mrharicot
I am getting strangely high error result when I test by the downloaded kitti model. The generated depthmaps both using the given-model as well as my own trained models all look good while their evaluation on kitti_stereo_2015_test_files gives me the same abnormal results...

Testing:
python monodepth_main.py --mode test --data_path path_to_kittiDataset/test_data/
--filenames_file path_to_kitti_dataset/kitti_stereo_2015_test_files.txt --log_directory $logdir
--checkpoint_path path_to_models/model_kitti --output_directory $outputdir

Evaluating:
python utils/evaluate_kitti.py --split kitti --predicted_disp_path $outputdir/disparities.npy --gt_path utils/data_scene_flow/test_data

-sample of generated depth:

• after
  min_depth = 1e-3 max_depth = 80 pred_depth[pred_depth < min_depth] = min_depth pred_depth[pred_depth > max_depth] = max_depth

-corresponding gt-depth:

This ends up with very large values in 'thresh' when computing errors.

Will really appreciate it if you can get me some hints @dantkz @gosip @Hirico .

Thanks, Ali

Originally posted by @aliko70 in #199 (comment)

[NovaMind-Z]

Hi, I use the pretrained kitti model to test a pciture in KITTI2015 and get the disparity map like this:

while the ground truth disparity map is as below :

Could you please tell me why they are quite different in their pixel value?

[ndsclark]

Hi @aliko70
Please check whether you have changed the format of image containing the ground truth from .png to .jpg. If so, please change it to .png.

[deffandchen]

Hi @aliko70
Please check whether you have changed the format of image containing the ground truth from .png to .jpg. If so, please change it to .png.
The ground truth in the original dataset is .png. Do you mean it doesn't need to be changed?

[ndsclark]

Please check whether you have changed the format of image containing the ground truth from .png to .jpg. If so, please change it to .png.
The ground truth in the original dataset is .png. Do you mean it doesn't need to be changed?

Yeah，training set need to be changed, but the ground truth needn't to be changed.

[deffandchen]

Please check whether you have changed the format of image containing the ground truth from .png to .jpg. If so, please change it to .png.
The ground truth in the original dataset is .png. Do you mean it doesn't need to be changed?

Yeah，training set need to be changed, but the ground truth needn't to be changed.

Thank you very much! I have got the right results.