Interpretation of Baseline Overview acc on Twibot-22 f1 on Twibot-22

[ohauglend]

Hi, as far as I understand the Baseline Overview acc and f1 are training measures where the model associated with the paper have been fitted on the whole data and then the measures are produced. I am wondering if I can compare these measures directly to a model trained on the Twibot-22 data, and if not then why so?

The word re-implemented is used consistently in the paper and on git. I understand this as re-trained, please let me know if I have misunderstood?

[BunsenFeng]

Hi, thank you for your interest in our work. Yes, you can compare the accuracy and F1-score numbers directly to a model trained on the TwiBot-22 data. By "reimplemented" we meant that not all baselines have available code, so we implemented baselines when necessary. All results in the "TwiBot-22" column indicate that the model is trained on the TwiBot-22 training set and evaluated performance on the TwiBot-22 test set.

Please let me know if you have more questions.

[ohauglend]

Thanks, this is really helpful, is there anywhere to access the Twibot-22 test set either through git or the drive folder?

[whr000001]

Hi, thank you for your interest in our work. The test set can be found in the 'split.csv' file. The first column denotes the user id and the second denotes this user belongs to train/val/test set.

[ohauglend]

Great, is it anyway to see what data is linked to Test 1 Test 2 and Test 3?

[TamSiuhin]

Hi @ohauglend! Test1 contains 1000 expert annotated users described in section 3.2. Test2 denotes 500 users in the annotation quality study (section A.5). Test 3 is the merge of test1 and test2.