Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

is there a multithreads method for add value? #52

Open
LzyloveRila opened this issue Apr 20, 2023 · 2 comments
Open

is there a multithreads method for add value? #52

LzyloveRila opened this issue Apr 20, 2023 · 2 comments

Comments

@LzyloveRila
Copy link

when add 1 billion values to the bloom filters, it cost nearly 4 hours on my server, I think it's possible to calculate hashes for batches of values in multithreads.

@mireklzicar
Copy link

My strategy was to create a lot of filters in parallel (lets say you have billion values; then you can create 1000 filters adding values to them in parallel) and then merge them using the .union() function.

@prashnts
Copy link
Owner

Multithreading won't necessarily help as adding/hashing is a CPU bound operation. @mireklzicar's approach is preferable.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants