Consider using ahocorasick-rs instead of pyahocorasick

[nijel]

Describe the problem

https://pypi.org/project/ahocorasick-rs/ seems faster alternative to pyahocorasick.

Describe the solution you'd like

It would be useful to benchmark it in Weblate use-case and switch to it in case it outperforms pyahocorasick.

Describe alternatives you've considered

No response

Screenshots

No response

Additional context

That being said, I've seen ahocorasick_rs run 1.5× to 7× as fast as pyahocorasick, depending on the options used.

[github-actions]

This issue seems to be a good fit for newbie contributors. You are welcome to contribute to Weblate! Don't hesitate to ask any questions you would have while implementing this.

You can learn about how to get started in our contributors documentation.

[nijel]

I've done some really basic benchmark and it seems that it's reasonable to switch:

from weblate.checks.data import IGNORE_WORDS
import ahocorasick
import ahocorasick_rs
import timeit


def build_py():
    automaton = ahocorasick.Automaton()
    for term in IGNORE_WORDS:
        automaton.add_word(term, term)
    automaton.make_automaton()
    return automaton


def build_rs():
    return ahocorasick_rs.AhoCorasick(
        IGNORE_WORDS,
        implementation=ahocorasick_rs.Implementation.ContiguousNFA,
        store_patterns=False,
    )


print("Build")
print(timeit.timeit("build_py", globals={"build_py": build_py}))
print(timeit.timeit("build_rs", globals={"build_rs": build_rs}))

ac_py = build_py()
ac_rs = build_rs()

print("Find")
print(
    timeit.timeit(
        "list(ac_py.iter('Please enter the correct username and password.'))",
        globals={"ac_py": ac_py},
    )
)

print(
    timeit.timeit(
        "ac_rs.find_matches_as_indexes('Please enter the correct username and password.')",
        globals={"ac_rs": ac_rs},
    )
)

[github-actions]

Thank you for your report; the issue you have reported has just been fixed.

• In case you see a problem with the fix, please comment on this issue.
• In case you see a similar problem, please open a separate issue.
• If you are happy with the outcome, don’t hesitate to support Weblate by making a donation.