Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Serializable PyTokenizer #58

Closed
mh-northlander opened this issue Oct 1, 2021 · 2 comments
Closed

Serializable PyTokenizer #58

mh-northlander opened this issue Oct 1, 2021 · 2 comments
Labels
python Python binding-related
Milestone
0.6

Comments

@mh-northlander
Copy link
Collaborator

mh-northlander commented Oct 1, 2021 •
edited
Loading

Part of #52, relates to #38.
Let PyTokenizer serializable so that we can save/load the model.
@mh-northlander
Copy link
Collaborator Author

This not necessary since we can reconstruct tokenizer using same configuration.

@eiennohito eiennohito added this to the 0.1 milestone Oct 1, 2021
@eiennohito eiennohito added the python Python binding-related label Oct 8, 2021
@mh-northlander
Copy link
Collaborator Author

Close this, since as mentioned above this feature is not necessary for the for now.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
python Python binding-related
Projects
None yet
Milestone
0.6
Development

No branches or pull requests
2 participants
@eiennohito @mh-northlander