WebquestionSP result lower than 0.714

[xianghuisun]

This is the experimental result on WebQSP(No code changes)
My question is that you said the WebQSP had 572 predicates in the paper.
And your code is

but the fbwq_full KG I got from https://drive.google.com/drive/folders/1RlqGBMo45lTmWz9MUPTq-0KcjSd3ujxc is

:
so should I experiment with relation_ids.del instead of relations.dict

Originally posted by @xianghuisun in #3 (comment)

[shijx12]

Hi, there should be 1144 relations actually. The number in the paper is wrong.

For the inconsistent results on WebQSP, we think it is due to the different versions of Transformers package and its BERT weights.
We are trying to recover our previous experimental environment. We conducted the experiments about one-year ago and our machine has been upgraded, so it may take some time.

We have provided the checkpoints in https://cloud.tsinghua.edu.cn/f/786b9853c1d840578025/?dl=1. You can use it for inference.

[Huiopfsdfsdf]

嗨，实际上应该有 1144 个关系。论文中的数字是错误的。

对于WebQSP上不一致的结果，我们认为这是由于不同版本的变压器包及其BERT权重造成的。我们正在努力恢复以前的实验环境。我们大约在一年前进行了实验，我们的机器已经升级，所以可能需要一些时间。

我们在 https://cloud.tsinghua.edu.cn/f/786b9853c1d840578025/?dl=1 提供了检查站。您可以将其用于推理。

Hello brother, is there a log of cwq? Thank you very much