Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

MOVE errors when autoPipeline=true and all commands are custom #1890

Open
jjsimps opened this issue May 3, 2024 · 1 comment
Open

MOVE errors when autoPipeline=true and all commands are custom #1890

jjsimps opened this issue May 3, 2024 · 1 comment

Comments

@jjsimps
Copy link

jjsimps commented May 3, 2024

A lot of the calls that I perform to redis are to the JSON redis module, and so I need to use the custom redis.call() function. e.g. client.call('JSON.SET', 'some_key', '$', ...).

However, I noticed that I got the following error rather sporadically:

ReplyError: MOVED 6734 10.16.0.3:31002
    at new Promise (<anonymous>)
    at executeWithAutoPipelining(...)
    ...

I ended up figuring out why this happens:

It seems like when autoPipeline=true and ALL commands sent in the pipeline are custom commands (i.e. only commands sent via client.call(...), the commands are sent to a random node instead.

This is because when the pipelineSlot is calculated based on the keys, it has no keys to reference, as everything is considered as args. This results in a random pipeline slot being chosen. Which is often wrong, so it causes the pipeline to fail.

If there is a single native command though (for example redis.set()), then it is able to determine which node to send it to correctly, and everything works fine. Also, if i set autoPipeline=false, then I get zero move errors. These two observations were made as I was debugging the library code.

It seems like it would need a better way to determine which node to send it to. Either allowing to specify the number of keys with client.call() or a some sort of heuristic that inspects the first arg and assumes that is the key (like it is done elsewhere in the code).

@jjsimps jjsimps changed the title Error when autoPipeline=true and all commands are custom MOVE errors when autoPipeline=true and all commands are custom May 3, 2024
@ezioda004
Copy link

ezioda004 commented May 3, 2024

I had faced this similar issue, and ended up putting all the JSON commands in autoPipeliningIgnoredCommands options. Ref.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants