Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Timeout awaiting response #2673

Open
MykolaKushnarenko opened this issue Mar 14, 2024 · 2 comments
Open

Timeout awaiting response #2673

MykolaKushnarenko opened this issue Mar 14, 2024 · 2 comments

Comments

@MykolaKushnarenko
Copy link

MykolaKushnarenko commented Mar 14, 2024

Hey team!

Now and then we encounter "Timeout awaiting response"
We're hosting our env on Azure and using Azure Cache for Redis
Tier: Standard 1 GB

Here is the error message:
Timeout awaiting response (outbound=0KiB, inbound=0KiB, 5884ms elapsed, timeout is 5000ms), command=GET, next: GET #, inst: 0, qu: 0, qs: 0, aw: False, bw: SpinningDown, rs: ReadAsync, ws: Idle, in: 0, last-in: 210, cur-in: 0, sync-ops: 0, async-ops: 18238, serverEndpoint: #, conn-sec: 22769.29, aoc: 0, mc: 1/1/0, mgr: 10 of 10 available, clientName: client(SE.Redis-v2.7.20.59961), IOCP: (Busy=0,Free=1000,Min=1,Max=1000), WORKER: (Busy=1,Free=32766,Min=1,Max=32767), POOL: (Threads=2,QueuedItems=0,CompletedItems=460840,Timers=10), v: 2.7.20.59961 (Please take a look at this article for some common client-side issues that can cause timeouts: https://stackexchange.github.io/StackExchange.Redis/Timeouts)

The issue occurred at 23:49:36.5374749

Server load:
image
CPU:
image
Cache hit:
image
Cache read:
image
Total keys:
image

Would it be at all possible to lend a helping hand? We would highly appreciate your help.
Thanks in advance!

@NickCraver
Copy link
Collaborator

This looks like the client is actually awaiting data. The first question that comes to mind is: how large are these values that the GET is fetching?

@MykolaKushnarenko
Copy link
Author

Hey @NickCraver I'm really grateful for your reply!

There are basically 2 scenarios when this issue is reproduced:

  1. GET for a health check (GET HEALTHCHECK).
  2. GET to fetch some stashed data.

In the first case, we're expecting just a response (nil) even though there is nothing stored by this key(Key: HEALTHCHECK). But still, we get a timeout from time to time.
SLOWLOG GET showed "GET HealthChech":
image

Regarding real data, the size of the value was about 4.5 KB, a simple array of strings.
It doesn't, however, occur all the time. It may happen a couple of times through 2 weeks period.
SLOWLOG GET showed:
image

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants