-
Notifications
You must be signed in to change notification settings - Fork 22
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
failed to replicate the result of top1, top5 acc for CVAE model #2
Comments
what is the performance of your Base model? |
Thanks for your reply. I got the acc result on top1/top5 as 0.304/0.543 for CVAE model from the rl_run.py as you will get the test set performance before training. Any ideas about the problem? Best |
This looks like the result of the Base model. Have you tried printing the acc results of your Base model? |
Thanks for your comment. I will try the base model and let you know the result. |
Sorry for the late reply. I ran the base model and the top1/top5 accuracy is 0.349/0.575, which is similar to what you reported in the paper. However, the CVAE model is even worse than the baseline seq2seq. Please let me know your opinion. Thanks! |
Hi! I got the same accuracy problem as you discussed here when replicating the model and i am still confused why.If you guys could give me some advice i will be very appreciated. |
@claude-zhou
|
|
I am thinking that we should use "infer_outputs" to calculate the CrossEntropy and calculate the perplexity further |
I will be appreciate it very much if you could give some advice @claude-zhou |
Hi,
I am trying to replicate your chatbot model. I did exactly the same thing as you mentioned in the README. I got more or less similar kl/reconstruction loss for training CVAE. But for emoji classification, top1 and top5 acc for CVAE model is only 30.4% and 54.3%, which is much worse than the results you reported in the paper. Can you give some suggestions about this?
Thanks
Peng
The text was updated successfully, but these errors were encountered: