Error while inference

[JawadTawhidi]

Hi, I cloned your code, I downloaded your pre-computed results of DAVIS2017 and DAVIS2016. I used your results for calculating J and F values and it worked.

But when I propagate the temporal information using your temporal propagation model on your precomputed image segmentations of DAVIS2016 and DAVIS2017 and then I want to calculate the J and F values, it is getting error, it seems it is not accepting the segmentations. This is error is coming:

Evaluating sequences for the unsupervised task...
Traceback (most recent call last):
File "evaluation_method.py", line 40, in
dataset_eval = DAVISEvaluation(davis_root=args.davis_path, task=args.task, gt_set=args.set, year=args.year)
File "/home/users/JawadT/davis2016/davis2017-evaluation/davis2017/evaluation.py", line 27, in init
self.dataset = DAVIS(root=davis_root, task=task, subset=gt_set, sequences=sequences, codalab=codalab, year=self.year)
File "/home/users/JawadT/davis2016/davis2017-evaluation/davis2017/davis.py", line 46, in init
self._check_directories()
File "/home/users/JawadT/davis2016/davis2017-evaluation/davis2017/davis.py", line 72, in _check_directories
raise FileNotFoundError(f'Annotations folder for the {self.task} task not found, download it from {self.DATASET_WEB}')
FileNotFoundError: Annotations folder for the unsupervised task not found, download it from https://davischallenge.org/davis2017/code.html

Note: I use this command for DAVIS2016:

python evaluation/eval_saliency.py --mask_path davis2016/Unsup-DAVIS16/Unsup-DAVIS16-DIS-detections --img_path davis2016/Davisfromsite/DAVIS/JPEGImages/480p --output output29 --imset_path davis2016/Davisfromsite/DAVIS/ImageSets/480p/val.txt

But when I take its results to calculate J and F values, it is giving the above error.(I am specifying the directories correctly）

[JawadTawhidi]

For DAVIS2017 I use this command:

python evaluation/eval_with_detections.py --mask_path davis2017/Unsup-DAVIS17/Unsup-DAVIS17-val-EntitySeg-detections --img_path davis2017/DAVIS/JPEGImages/480p --dataset u
nsup_davis17 --temporal_setting semionline --output output2017third --chunk_size 4

[hkchengrex]

FileNotFoundError: Annotations folder for the unsupervised task not found

I think this line informs you of the error.

[JawadTawhidi]

But the fact is that I am setting the annotations directory correctly. This is confusing me why it is giving this kind of error while the directories are correct.

[hkchengrex]

If you think the annotation directory is set correctly but the code is telling you otherwise, it might be a bug in the evaluation script. The DAVIS challenge people might have a better answer than I do.

[JawadTawhidi]

I am thinking maybe the way I propagate the temporal information is not correct. I use your precomputed segmentations, I downloaded DAVIS17 and DAVIS16 from its official site, I use the above commands which I added in previous comments.

[JawadTawhidi]

In addition, when I use your final results, The evaluation script is not giving any error. But when I put my final results that error is coming.

[hkchengrex]

The fact that the code is asking you to download the dataset instead of producing a different type of error makes me think that that is highly unlikely. I used a similar command and it worked just fine. Do note that the dataset for the semi-supervised task and the unsupervised task is different.

[hkchengrex]

In addition, when I use your final results, The evaluation script is not giving any error. But when I put my final results that error is coming.

DAVIS evaluation script uses the cached results (.csv) in the output.

[JawadTawhidi]

I am really confused. Let me try again and again.

[JawadTawhidi]

The fact that the code is asking you to download the dataset instead of producing a different type of error makes me think that that is highly unlikely. I used a similar command and it worked just fine. Do note that the dataset for the semi-supervised task and the unsupervised task is different.

Yes. I considered the differences between the datasets of Semi-supervised and Unsupervised task.

[JawadTawhidi]

Hi, I solved the problem for DAVIS2017. But for DAVIS 2016 still I am getting the same error, maybe it is because of the changes which I made to the code to evaluate davis16. I used the code which you mentioned me here: davisvideochallenge/davis2017-evaluation#4

I just want to know that except the changes the guy mentioned here davisvideochallenge/davis2017-evaluation#4, do we need more changes in other files for davis16?

[hkchengrex]

As mentioned, I recommend using the script listed in our documentation: https://github.com/hkchengrex/Tracking-Anything-with-DEVA/blob/main/docs/EVALUATION.md.

If you would like to implement it yourself, I'm afraid that you would have to figure it out yourself.

[JawadTawhidi]

Ok. It seems for davis2016 you suggest this one: https://github.com/hkchengrex/vos-benchmark

Am I right?

[hkchengrex]

Correct.