Random failure on chain.blocks event in silverback

[johnson2427]

Environment information

• ape and plugin versions:

$ ape --version
# 0.8.14

$ ape plugins list
  alchemy      0.8.2
  etherscan    0.8.3
  vyper        0.8.5

• Python Version: 3.11.10
• OS: linux (Ubuntu 22.04.4)

What went wrong?

The repository that this failed in is public: https://github.com/johnson2427/erc-4626
silverback run "bots.silverback_yield:app" from the docker container.
On random chain.blocks events, I will receive this failure. The block that was created would then be skipped and not included in the output data. The stdout should look like:

app-1  | SUCCESS: update_shareprice[block_number=20794117,block_hash=0xfb0807fb179f42d3ff7854ae78b445e3d5acffbaa25cf0206efb9209760024af] - 0.210s (1.6%)
app-1  | Price Event: 0.9562281424937263
app-1  | SUCCESS: update_shareprice[block_number=20794118,block_hash=0xb9231027e924306bdb5fe4449aa36860d9740548298bb02244bcdee13bfc4073] - 0.250s (1.9%)
app-1  | Price Event: 0.9562281033652879
app-1  | SUCCESS: update_shareprice[block_number=20794119,block_hash=0x9dc65c0e1da0f4a8d4785d4e0ce9341dba837e43b39eb3c5d84eeae8547fd4c2] - 0.210s (1.6%)
app-1  | Price Event: 0.9562281033652879
app-1  | SUCCESS: update_shareprice[block_number=20794120,block_hash=0xbaf46d786644d23d65d34fdbda07d06af42c6a98872960e2258bae0cae3485af] - 0.220s (1.7%)
app-1  | Price Event: 0.9562281033652879
app-1  | SUCCESS: update_shareprice[block_number=20794121,block_hash=0x7bdff23e1e1638b08e42803e6b7eb1f27d1b4602736542e8a6a0ac21208370a2] - 0.170s (1.3%)

When this error occurs, the block that caused the failure would not appear in the stdout after restart (expected behavior if there is a failure)

Considering this is only printing the log information, I'm not 100% why these random failures occur.

app-1  | Task exception was never retrieved
app-1  | future: <Task finished name='Task-10' coro=<WebsocketRunner._event_task() done, defined at /usr/local/lib/python3.11/site-packages/silverback/runner.py:301> exception=ConnectionClosedError(None, None, None)>
app-1  | Traceback (most recent call last):
app-1  |   File "/usr/local/lib/python3.11/site-packages/websockets/legacy/protocol.py", line 963, in transfer_data
app-1  |     message = await self.read_message()
app-1  |               ^^^^^^^^^^^^^^^^^^^^^^^^^
app-1  |   File "/usr/local/lib/python3.11/site-packages/websockets/legacy/protocol.py", line 1033, in read_message
app-1  |     frame = await self.read_data_frame(max_size=self.max_size)
app-1  |             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
app-1  |   File "/usr/local/lib/python3.11/site-packages/websockets/legacy/protocol.py", line 1108, in read_data_frame
app-1  |     frame = await self.read_frame(max_size)
app-1  |             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
app-1  |   File "/usr/local/lib/python3.11/site-packages/websockets/legacy/protocol.py", line 1165, in read_frame
app-1  |     frame = await Frame.read(
app-1  |             ^^^^^^^^^^^^^^^^^
app-1  |   File "/usr/local/lib/python3.11/site-packages/websockets/legacy/framing.py", line 68, in read
app-1  |     data = await reader(2)
app-1  |            ^^^^^^^^^^^^^^^
app-1  |   File "/usr/local/lib/python3.11/asyncio/streams.py", line 748, in readexactly
app-1  |     raise exceptions.IncompleteReadError(incomplete, n)
app-1  | asyncio.exceptions.IncompleteReadError: 0 bytes read on a total of 2 expected bytes
app-1  |
app-1  | The above exception was the direct cause of the following exception:
app-1  |
app-1  | Traceback (most recent call last):
app-1  |   File "/usr/local/lib/python3.11/site-packages/silverback/runner.py", line 319, in _event_task
app-1  |     async for raw_event in self.subscriptions.get_subscription_data(sub_id):
app-1  |   File "/usr/local/lib/python3.11/site-packages/silverback/subscriptions.py", line 135, in get_subscription_data
app-1  |     await self.__anext__()
app-1  |   File "/usr/local/lib/python3.11/site-packages/silverback/subscriptions.py", line 49, in __anext__
app-1  |     return await self._receive()
app-1  |            ^^^^^^^^^^^^^^^^^^^^^
app-1  |   File "/usr/local/lib/python3.11/site-packages/silverback/subscriptions.py", line 58, in _receive
app-1  |     message = await asyncio.wait_for(self.connection.recv(), timeout)
app-1  |               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
app-1  |   File "/usr/local/lib/python3.11/asyncio/tasks.py", line 452, in wait_for
app-1  |     return await fut
app-1  |            ^^^^^^^^^
app-1  |   File "/usr/local/lib/python3.11/site-packages/websockets/legacy/protocol.py", line 568, in recv
app-1  |     await self.ensure_open()
app-1  |   File "/usr/local/lib/python3.11/site-packages/websockets/legacy/protocol.py", line 939, in ensure_open
app-1  |     raise self.connection_closed_exc()
app-1  | websockets.exceptions.ConnectionClosedError: no close frame received or sent
app-1  | Task exception was never retrieved
app-1  | future: <Task finished name='Task-11' coro=<WebsocketRunner._event_task() done, defined at /usr/local/lib/python3.11/site-packages/silverback/runner.py:301> exception=ConnectionClosedError(None, None, None)>
app-1  | Traceback (most recent call last):
app-1  |   File "/usr/local/lib/python3.11/site-packages/websockets/legacy/protocol.py", line 963, in transfer_data
app-1  |     message = await self.read_message()
app-1  |               ^^^^^^^^^^^^^^^^^^^^^^^^^
app-1  |   File "/usr/local/lib/python3.11/site-packages/websockets/legacy/protocol.py", line 1033, in read_message
app-1  |     frame = await self.read_data_frame(max_size=self.max_size)
app-1  |             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
app-1  |   File "/usr/local/lib/python3.11/site-packages/websockets/legacy/protocol.py", line 1108, in read_data_frame
app-1  |     frame = await self.read_frame(max_size)
app-1  |             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
app-1  |   File "/usr/local/lib/python3.11/site-packages/websockets/legacy/protocol.py", line 1165, in read_frame
app-1  |     frame = await Frame.read(
app-1  |             ^^^^^^^^^^^^^^^^^
app-1  |   File "/usr/local/lib/python3.11/site-packages/websockets/legacy/framing.py", line 68, in read
app-1  |     data = await reader(2)
app-1  |            ^^^^^^^^^^^^^^^
app-1  |   File "/usr/local/lib/python3.11/asyncio/streams.py", line 748, in readexactly
app-1  |     raise exceptions.IncompleteReadError(incomplete, n)
app-1  | asyncio.exceptions.IncompleteReadError: 0 bytes read on a total of 2 expected bytes
app-1  |
app-1  | The above exception was the direct cause of the following exception:
app-1  |
app-1  | Traceback (most recent call last):
app-1  |   File "/usr/local/lib/python3.11/site-packages/silverback/runner.py", line 319, in _event_task
app-1  |     async for raw_event in self.subscriptions.get_subscription_data(sub_id):
app-1  |   File "/usr/local/lib/python3.11/site-packages/silverback/subscriptions.py", line 135, in get_subscription_data
app-1  |     await self.__anext__()
app-1  |   File "/usr/local/lib/python3.11/site-packages/silverback/subscriptions.py", line 49, in __anext__
app-1  |     return await self._receive()
app-1  |            ^^^^^^^^^^^^^^^^^^^^^
app-1  |   File "/usr/local/lib/python3.11/site-packages/silverback/subscriptions.py", line 58, in _receive
app-1  |     message = await asyncio.wait_for(self.connection.recv(), timeout)
app-1  |               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
app-1  |   File "/usr/local/lib/python3.11/asyncio/tasks.py", line 452, in wait_for
app-1  |     return await fut
app-1  |            ^^^^^^^^^
app-1  |   File "/usr/local/lib/python3.11/site-packages/websockets/legacy/protocol.py", line 568, in recv
app-1  |     await self.ensure_open()
app-1  |   File "/usr/local/lib/python3.11/site-packages/websockets/legacy/protocol.py", line 939, in ensure_open
app-1  |     raise self.connection_closed_exc()
app-1  | websockets.exceptions.ConnectionClosedError: no close frame received or sent
app-1  | Closing app

How can it be fixed?

I am currently unsure of what the cause of the error is. It looks like there is an unexpected network issue, which may be the problem here? I might have some network latency issue that is causing the bytes to not be received, but thought this should be brought up just in case.

[linear]

SBK-557 Random failure on chain.blocks event in silverback

[fubuloubu]

this is related to websocket failure #84