You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Describe the bug
I'm using django-clickhouse-backend and want to work with clickhouse database using multiple python processes. For that purpose I need to close db connections before spawning new processes by calling
The problem is that if I've already queried DB in the main process before closing connections, I have the following warnings + exception:
clickhouse_driver.connection - WARNING - Error on clickhouse.***:9000 ping: Unexpected EOF while reading bytes
clickhouse_driver.connection - WARNING - Error on clickhouse.***:9000 ping: Unexpected EOF while reading bytes
...
clickhouse_driver.connection - WARNING - Connection was closed, reconnecting.
clickhouse_driver.connection - WARNING - Connection was closed, reconnecting.
...
django.db.utils.OperationalError: Code: 100. Unknown packet 4 from server clickhouse.***:9000
It seems that pings are not properly handled at connection reopening.
To Reproduce
Not a complete code, but general idea:
fromdatetimeimportdatetimefrommultiprocessingimportPoolfromdjango.dbimportconnectionsdefworker(interval):
"""get objects from clickhouse database for a given time interval"""objs=clickhouse_model.objects.filter(
created_at__gte=interval.from,
created_at__lt=interval.to,
)
defmain():
"""spawn multiple workers over time intervals"""min_date=clickhouse_model.objects.all.aggregate(min=Min('created_at'))['min']
intervals=get_invervals(from=min_date, to=datetime.now(), size=86400)
connections.close_all() # Django will automatically reopen connection on the first querywithPool(processes=processes) aspool:
for_inpool.imap_unordered(worker, intervals):
pass
Expected behavior
No warnings issued on user-initiated disconnect.
Pings are properly stopped.
I do not know why ping packets from closed connection are received on reopen, but they have to be silently discarded.
Describe the bug
I'm using django-clickhouse-backend and want to work with clickhouse database using multiple python processes. For that purpose I need to close db connections before spawning new processes by calling
The problem is that if I've already queried DB in the main process before closing connections, I have the following warnings + exception:
It seems that pings are not properly handled at connection reopening.
To Reproduce
Not a complete code, but general idea:
Expected behavior
Versions
The text was updated successfully, but these errors were encountered: