Repositories list

• SaGe

  Public
  Code for SaGe subword tokenizer (EACL 2023)

  Python

  •

  MIT License

  •4•22•2•0•Updated Nov 30, 2024Nov 30, 2024

• tokenizers_intrinsic_benchmark

  Public
  Code for the paper "Greed is All You Need: An Evaluation of Tokenizer Inference Methods"

  Python

  •

  MIT License

  •0•6•0•0•Updated Nov 26, 2024Nov 26, 2024

• MeLeL-BiVERT

  Public
  MT evaluation - Carinne Thesis

  Jupyter Notebook

  •

  MIT License

  •0•0•0•0•Updated Nov 19, 2024Nov 19, 2024

• Privacy-Preserving-Token-Manipulation

  Public
  Python

  •0•0•0•0•Updated Jun 26, 2024Jun 26, 2024