-
Notifications
You must be signed in to change notification settings - Fork 49
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
queue length for SQLiteQueue is incorrect when running in multiple processes #76
Comments
thanks for reporting, i haven't tested the queue in multi-process environment. if you find further issue. let me know, i am happy to add the support . |
Can confirm, this is indeed happened. However, this causes a somewhat annoying/strange issue. If I push data into a FIFOSQLiteQueue with one process, the other one takes multiple seconds before actually fetching the values. |
@ThunderRush As far as I understand, using get on an empty queue is supposed to wait until it actually can get something. If you don't want this behaviour - you can either pass |
The multi-process support was not added in this lib currently, I would be happy if anyone has interest in providing this ability. @melbaa @ThunderRush @bavaria95 |
Recent updates have added |
Yes, this seems to be a bug. So instead of
use the workaround
|
Possibly expected behavior, but I think it's worth reporting, because the queue looks usable otherwise.
The queue size is set only once on queue creation.
self.total = self._count()
, so if we have a producer in 1 process and a consumer in another process, we end up with size in the negatives.To reproduce, we need producer and a consumer that's faster than the producer.
Calling q._count() returns the correct size, because it hits the DB, of course.
The text was updated successfully, but these errors were encountered: