Redis Pipelining Support

[leeaaron629]

Hi, I would like to know if Redis Pipelining will be supported in the future. This would be great for reducing network costs 👍

[gkorland]

@onlyhope yes we plan to add support for Pipeline in the future.

[lpellegr]

I just experienced the same need. It would be a very convenient feature that avoids writing Lua scripts.

[leeaaron629]

@gkorland Wondering how much performance benefits I would see for bulk ingestion between the following methods:

• TS.MADD (Sync)
• TS.MADD (Async)
• EVAL (LUA Script) / Pipelining

From my understanding TS.MADD is very similar to a bulk insertion with 1 network call? It can block incoming read requests if the bulk insertion is too large. TS.MADD (Async) is slightly faster because we are not waiting for a response, so we can send the next write requests without waiting. TS.EVAL is what Redis Pipeline uses under the hood? (Correct me if I am wrong here) All it does is evaluate a bunch of Redis Commands within the given script and is very similar to TS.MADD (Async).

Is the reason that Redis Pipelining is not supported because Redis Pipeline's performance is similar to a TS.MADD (Async) or a very large TS.MADD (Sync). The reason to not do a large TS.MADD (Sync) is because it can block incoming requests and there's a limit in data set size.

So if I understand correctly TS.MADD (Async) would be sufficient for bulk inserts?

EDITED:

I realized async calls are supported client-side in Lettuce, so pipelining is the way to go.

Here's a way to do it without EVAL by using Jedis API

fun pipeline(commands: Pipeline.() -> Unit) {
        jedisPool.resource.use {
            val pipe = it.pipelined()
            // sendCommand(Command.MADD, *tsdbData)
            pipe.commands()
            pipe.sync()
        }
    }