Difference between Redis pub/sub and MQTT

[neerajasjawali]

Hi all,
Please help me understand the difference between using Redis pub/sub and MQTT as EdgeX MessageBus.

Thanks in advance.

[lenny-goodell]

At a high level, Redis Pub/Sub is a basic fire and forget in memory pub/sub implementation while MQTT is a more robust implementation that is more resilient and handles heavy loads better. Redis Pub/Sub doesn't require an addition broker since it uses the Redis instance already running for the Database. MQTT requires the additional MQTT Broker service to be running.

[neerajasjawali]

@lenny-intel I was working on an application where I had to send data of aprox 50MB. Neither the Redis pub/sub nor the MQTT was able to send the data. My system cache memory was being overused. What would be the possible reason? Or is it that EdgeX is not capable of handling such heavy load?

And it is said that Redis pub/sub is secure MessageBus, where as MQTT as secure MessageBus is yet to be implemented. When running in secure mode the Redis database service is secured with a username/password. Is this the only reason which makes it secure or are there more reasons to consider it as secure?

[bnevis-i]

And it is said that Redis pub/sub is secure MessageBus, where as MQTT as secure MessageBus is yet to be implemented. When running in secure mode the Redis database service is secured with a username/password. Is this the only reason which makes it secure or are there more reasons to consider it as secure?

No value judgement is being made as to whether redis is more secure than mosquitto or vice-versa, and security is a relative term that depends on what kind of adversaries you have in scope, the configuration of your system architecture, et cetera.

In context, EdgeX offers a no-security mode where no usernames/passwords of any kind are required for internal service communication (like to redis, consul, mqtt brokers, etc). In this context, "secure mode" means running redis in an authentication-required (username/password mode). From the perspective of an attacker sitting on network eavesdropping on database communication, username/password authentication doesn't protect against eavesdropping attacks, MITM attacks on the data stream, or maliciously dropping packets.

[bnevis-i]

Redis pubsub has a 32 MB limit and MQTT has an upper 260MB limit

Its hard to say why you are having memory issues offhand as things like excessive buffering of the data (i.e. trying to read the entire message into memory all at once) can balloon memory usage and cause issues if you don't have enough system or process memory (32-bit application?)

MQTT client authentication is supported in the latest EdgeX release, for both username password and tls client authentication (also supports anonymous with just server side TLS auth based on a CA certificate).

Note that the current EdgeX security policy is that it is left up to the adopter to secure network communication that is leaving the host. EdgeX does not implement TLS between any of its microservices as, among other reasons, it introduces a key management problem that can't be solved internally. (For example, when a TLS cert expires, should EdgeX terminate to pick up new keying material?) The one exception is that the API gateway has an untrusted TLS certificate, which is guaranteed to be untrusted and supposed to be replaced with an adopter-specified certificate.

[neerajasjawali]

Redis Pub/Sub limit can be extended. So what's the max limit?
I am working on a 64-bit RISC-V board (8GB memory).

So, we can say that using redis or MQTT as MessagesBus depends on the applications memory requirement.

[cloudxxx8]

Neither Redis pub/sub or MQTT should handle the 50MB streaming data. Your application service should connect to the streaming source directly, such as using RSTP for an USB camera.

[neerajasjawali]

Thank you for helping me understand the EdgeX MessageBus @cloudxxx8

Could you let me know what's the max redis pub/sub memory limit?