Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Feature] end-to-end test with CrossWOZ #10

Open
nuance1979 opened this issue May 12, 2020 · 1 comment
Open

[Feature] end-to-end test with CrossWOZ #10

nuance1979 opened this issue May 12, 2020 · 1 comment

Comments

@nuance1979
Copy link

Describe the feature
I was able to run python tests/test_BERTNLU-RuleDST-RulePolicy-TemplateNLG.py and got the same results as in README. Thanks a lot!

However, as soon as I tried to run the same test for CrossWOZ, I ran into the issue that there was no evaluator for CrossWOZ dataset. In fact, currently analyzer only works with the default MultiWOZ evaluator.

I tried to use MultiWOZ evaluator but it didn't work. I looked at CrossWOZ repo but it seemed there wasn't an end-to-end evaluator class, either.

Expected behavior
A working end-to-end evaluator for CrossWOZ dataset.

Additional context
n/a

@zqwerty
Copy link
Member

zqwerty commented May 13, 2020

Yes, we will add it in the future

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants