Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Binary pretrained model can't train a multi-class classifier? #172

Open
Bai1026 opened this issue Jul 28, 2023 · 0 comments
Open

Binary pretrained model can't train a multi-class classifier? #172

Bai1026 opened this issue Jul 28, 2023 · 0 comments

Comments

@Bai1026
Copy link

Bai1026 commented Jul 28, 2023

I use model_checkpoint = "clefourrier/graphormer-base-pcqm4mv2" # pre-trained model from which to fine-tune to be my pretrained model, and my dataset is here.
I follow the code in this website.
But change the

model = GraphormerForGraphClassification.from_pretrained(
    model_checkpoint, 

    # We have 167 attack patterns and 1 benign
    num_classes=168, 

    # provide this in case you're planning to fine-tune an already fine-tuned checkpoint
    ignore_mismatched_sizes = True, 

Since I have 168 class to be classified.
Then when I run the python script, I encountered:

Some weights of GraphormerForGraphClassification were not initialized from the model checkpoint at clefourrier/graphormer-base-pcqm4mv2 and are newly initialized because the shapes did not match:
- classifier.classifier.weight: found shape torch.Size([1, 768]) in the checkpoint and torch.Size([168, 768]) in the model instantiated
You should probably TRAIN this model on a down-stream task to be able to use it for predictions and inference.
  0%|                                                                                                                                                                                                                                          | 0/23120 [00:00<?, ?it/s]

For about 20 min, the log didn't move at all.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant