Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

I am getting this issue if anyone have solution or advice please reply #2797

Open
Akashappinfo opened this issue Sep 18, 2024 · 1 comment
Open

Comments

@Akashappinfo
Copy link

[FTL] Ln 440 Deviceld: 14519 Message : The message timed out in the backlog attempting to send because no connection became available,
command=GET, timeout: 5008, outbound: OKiB, inbound: 4KiB, inst: @, qu: 171, gs: @, aw: True, bw: CheckingForTimeoutComplete, rs: ReadAsync, ws: Idle, in: ©, in-pipe: @, out-
pipe: ©, last-in: 10, cur-in: @, sync-ops: 146873816, async-ops: ©, serverEndpoint: 10.75.101.232:8080, conn-sec: 456033.61, aoc: ©, mc: 1/1/@, mgr: 10 of 10 available,
clientName: charon(SE.Redis-v2.7.17.27858), T0CP: (Busy=0,Free=1008,Min=180,Max=1000), WORKER: (Busy=170,Free=32597,Min=108,Max=32767), POOL: (Threads=170,QueuedItems=
47,CompletedItems=697381603,Timers=232), v: 2.7.17.27058 (Please take a look at this article for some common client-side issues that can cause timeouts:
https: //stackexchange. github. io/StackExchange.Redis/ Timeouts)

@NickCraver
Copy link
Collaborator

If we look at sync-ops: 146873816 and WORKER: (Busy=170,Free=32597,Min=108,Max=32767), we can see that it appears all of the operations against Redis (an inherently long thing, travelling over the network in most cases) are synchronous instead of async. This is at least contributing to thread pool exhaustion which will result in starvation and timeouts.

I'd recommend moving all off-box operations like Redis and others to async.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants