Code

Return original text as tokens#1

Code

Merged

johnmbw merged 5 commits intoBrandwatchLtd:masterBrandwatchLtd/vnTokenizer:masterfrom johnmbw:return-original-text-as-tokensjohnmbw/vnTokenizer:return-original-text-as-tokensCopy head branch name to clipboardOct 25, 2018

+97-9

Commits

Commits on Oct 23, 2018

• Simple unit test for tokenization

  johnmbw

  committed
  e368013

  Copy full SHA for e368013

• very minimal README

  johnmbw

  committed
  d3958ee

  Copy full SHA for d3958ee

• Keep track of original text when normalising

  Show description for 4ddefc9
  johnmbw

  committed
  4ddefc9

  Copy full SHA for 4ddefc9

• minimal Travis CI file

  johnmbw

  committed
  a65fdbc

  Copy full SHA for a65fdbc

Commits on Oct 25, 2018

• Set to version 5.2.1-bw

  johnmbw

  committed
  2c8fe6c

  Copy full SHA for 2c8fe6c