Skip to content

Releases: susanhuhu/go-sentencepiece-encoder

Fixed perf issue further

20 Jun 13:34
Compare
Choose a tag to compare
v0.0.4

refined implementation and fixed perf issue

Fixed perf issue

16 Jun 19:04
Compare
Choose a tag to compare
v0.0.3

refined implementation and fixed perf issue

v0.0.2

05 May 00:43
Compare
Choose a tag to compare

Fixed a corner case where first padding itself is treated as a token

v0.0.1

22 Apr 00:37
Compare
Choose a tag to compare
Exposed token offsets and make sure the offset matched original offset

in text