MQ empty annotation

[dairui01]

Hello,

I face the following issue while evaluating the prediction:

---

3. Retrieval evaluation starts!

---

a. Generate retrieval!
joblib.externals.loky.process_executor._RemoteTraceback:
"""
Traceback (most recent call last):
File "/home/rdai/anaconda3/lib/python3.7/site-packages/joblib/externals/loky/process_executor.py", line 431, in _process_worker
r = call_item()
File "/home/rdai/anaconda3/lib/python3.7/site-packages/joblib/externals/loky/process_executor.py", line 285, in call
return self.fn(*self.args, **self.kwargs)
File "/home/rdai/anaconda3/lib/python3.7/site-packages/joblib/_parallel_backends.py", line 593, in call
return self.func(*args, **kwargs)
File "/home/rdai/anaconda3/lib/python3.7/site-packages/joblib/parallel.py", line 253, in call
for func, args, kwargs in self.items]
File "/home/rdai/anaconda3/lib/python3.7/site-packages/joblib/parallel.py", line 253, in
for func, args, kwargs in self.items]
File "/data/stars/user/rdai/PhD_work/Ego4d/code/episodic-memory/MQ/Evaluation/ego4d/generate_retrieval.py", line 88, in _gen_retrieval_video
df = rm_other_category(df, test_anno['annotations'], classes)
File "/data/stars/user/rdai/PhD_work/Ego4d/code/episodic-memory/MQ/Evaluation/ego4d/generate_retrieval.py", line 73, in rm_other_category
df_v = pd.concat(df_v)
File "/home/rdai/anaconda3/lib/python3.7/site-packages/pandas/core/reshape/concat.py", line 225, in concat
copy=copy, sort=sort)
File "/home/rdai/anaconda3/lib/python3.7/site-packages/pandas/core/reshape/concat.py", line 259, in init
raise ValueError('No objects to concatenate')
ValueError: No objects to concatenate

Then I check the clip_annotations.json generated by the Convert_annotations.py is empty in the 'annotations'.
I am predicting with the Slowfast feature from ego4d and the baseline code in this repo.

[fedegonzal]

Hey, news about this error? I'm facing on the same problem. Thanks!

[coolbay]

It looks like you are using a different version of the annotation file, which has been updated several times. Could you please provide the annotation file (input to the Convert_annotations.py) you are using?

[fedegonzal]

I used this command to download annotations:
ego4d --output_directory="./data" --datasets annotations --benchmarks EM --version v2

You can find my annotations folder here:
https://drive.google.com/drive/folders/1-BW-ZkFiDoodzRjKFvRwBgFfgH5vzLTQ?usp=sharing

Later I'm running:

1. Convert_annotation.py
2. Train.py --use_xGPN --is_train true --dataset ego4d --feature_path data/v2/slowfast8x8_r101_k400/ --checkpoint_path data/v2/moments_models/ --clip_anno data/v2/annotations/clip_annotations.json --moment_classes data/v2/annotations/moment_classes_idx.json --batch_size 4 --train_lr 0.0001
3. Infer.py --use_xGPN --is_train false --dataset ego4d --feature_path data/v2/slowfast8x8_r101_k400/ --checkpoint_path data/v2/moments_models/ --moment_classes data/v2/annotations/moment_classes_idx.json --clip_anno data/v2/annotations/clip_annotations.json --output_path inferences/ --batch_size 4 --infer_datasplit test
4. Eval.py --output_path inferences/ --clip_anno data/v2/annotations/clip_annotations.json --moment_classes data/v2/annotations/moment_classes_idx.json --eval_stage all --infer_datasplit test

Eval should generate two files:

• detections_postNMS.json seems to be fine (but without annotations at the final part)
• retrieval_postNMS.json (facing on an error because it can't found annotations in the previous file)

If you need something else, just tell me. I want to resolve it soon as possible! Thanks

[coolbay]

@fedegonzal

I tried running "Convert_annotation.py" using your annotation files "moments_train.json", and "moments_val.json". But I didn't see the [info_path] "ego4d.json" in your folder, so I prepared it myself. Doing so, the output annotation file "clip_annotations.json" I got is fine and not empty.

1. Can you check if you got an empty "clip_annotations.json" from your Step 1?
2. If so, may I ask if you have used all the input files required by "Convert_annotation.py"? 1) [feat_path], 2)[info_path], 3)[annot_path_train], [annot_path_val], [annot_path_test].

[fedegonzal]

This is my generated clip annotations https://drive.google.com/file/d/1qLFFn0wnYIsWCOUEEi1MHMRj4DglQUSP/view?usp=drivesdk thank you by supporting me! El El dom, 30 de abr. de 2023 a la(s) 09:56, Cathy ***@***.***> escribió:

…

@fedegonzal <https://github.com/fedegonzal> I tried running "Convert_annotation.py" using your annotation files "moments_train.json", and "moments_val.json". But I didn't see the [info_path] "ego4d.json" in your folder, so I prepared it myself. Doing so, the output annotation file "clip_annotations.json" I got is fine and not empty. 1. Can you check if you got an empty "clip_annotations.json" from you Step 1? 2. If so, may I ask if you have used all the input files required by "Convert_annotation.py"? 1) [feat_path], 2)[info_path], 3)[annot_path_train], [annot_path_val], [annot_path_test]. — Reply to this email directly, view it on GitHub <#36 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAGJSBWZ2Q5N72PAJMIQJDTXDYLLJANCNFSM6AAAAAAQOPKXCM> . You are receiving this because you were mentioned.Message ID: ***@***.***>

[fedegonzal]

Hi @coolbay I couldn't fix my problem

This is my retrieval_postNMS.json result, just empty values:

{ "375e2d5b-31fa-4d21-8aa9-8bdfba124681": [], 
  "365d86c5-3b3c-4fc5-82e0-948b2807b32e": [], 
  ...
}

I tried using the official EM model from the github repositry. I downloaded again annotations, slowfast features, I ran again Convert_annotation.py, Infer.py, but I have everytime the same empty result.

To run Convert_annotation.py I'm using this files as input:

• data/ego4d.json
• data/v2/slowfast8x8_r101_k400/
• data/v2/annotations/moments_train.json
• data/v2/annotations/moments_val.json
• data/v2/annotations/moments_test_unannotated.json

I must to use moments_test_unannotated.json that's right?

My clip_annotations.json file is here:
https://drive.google.com/file/d/1GT3yW2mc8opF9s1bGurBN1S6emrZ7XIJ/view?usp=share_link

And here the moment_classes_idx.json file:
https://drive.google.com/file/d/1-NANZXoF6KzjZKzaUEeu7rvDYjo55CRL/view?usp=share_link

Later I'm using this command to Infer.py (may be there is the problem, I don't know):

python episodic-memory/MQ/Infer.py \
  --use_xGPN \
  --is_train false \
  --dataset ego4d --feature_path data/v2/slowfast8x8_r101_k400/ \
  --checkpoint_path data/v2/moments_models_baseline/ \
  --moment_classes data/v2/annotations/moment_classes_idx.json \
  --clip_anno data/v2/annotations/clip_annotations.json  \
  --output_path inferences/ \
  --batch_size 4 \
  --infer_datasplit test

And this for Eval.py

python episodic-memory/MQ/Eval.py \
  --output_path inferences/ \
  --clip_anno data/v2/annotations/clip_annotations.json \
  --moment_classes data/v2/annotations/moment_classes_idx.json \
  --eval_stage all \
  --infer_datasplit test \
  --use_xGPN

Thanks for your time!