-
Notifications
You must be signed in to change notification settings - Fork 46
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add sidekiq-throttler to sidekiq jobs involving API #256
Comments
This would address #96 as well I guess? 📕 |
Yes I'd hope so :D However, sidekiq-throttler looks like it's not supported anymore. sidekiq-rate-limiter looks a bit wonky and unsupported, too. There's a sidekiq4 branch which looks like it works: https://github.com/Docady/sidekiq-rate-limiter/tree/sidekiq-4 |
Sidekiq Enterprise has Rate Limiting. No clue how much this would cost... We can ask them, though. |
One problem is that sidekiq-limit_fetch, which we're currently using, is incompatible with the other rate-limiting gems, since they all patch into the way Sidekiq retrieves jobs from the queue. One workaround is to remove sidekiq-limit_fetch and use the time-limits for non-API jobs like parsing, too. Yay/Nay? |
As in this comment thread: #251 (comment)
Use https://github.com/gevans/sidekiq-throttler for all API jobs to be nicer ✨ 💖 ✨
PLOS has 7200 requests per 24 hours, Mendeley has 500/hour maybe outdated source, Fitbit 150/hour source, SNPedia ???, but something much less
Edit: Looks like sidekiq-throttler only supports Sidekiq 2/3, not 4!
The text was updated successfully, but these errors were encountered: