Request support of MQTT Last Will and Testament (LWT)

[ftylitak]

Issue

Since the introduction of the mproxy in Mainflux (and now Magistrala), it lacks support of listening MQTT Last Will messages from MQTT broker.

Case

We have the case of devices reporting connection status everytime the connect/disconnect to the network through a simple MQTT publish. To cover edge cases of devices that are forcefully unplugged from the power, we also setup Last Will on every connection to report that the device is disconnected if not properly terminated.

This worked perfectly with Mainflux 0.10.0 that we have been using for a while, though after upgrading to mainflux 0.12.0 that has been using mproxy, these messages we not received.

Investigation results

After investigating the source code of mproxy, we ended up to two keypoints that prevent this feature to function with the current mproxy architecture.

a) at the two streams of each session, only messages comming from client to the broker are notified (passed onto nats). (https://github.com/absmach/mproxy/blob/695f58d860bd1c588e106c9ad679a9cfb9b69dd2/pkg/session/stream.go#L77)

b) the Last Will messages are generated from the MQTT broker only if there are active subscribers at that time. Since the mproxy is only packet forwarer, it is not a subscriber so in the general case, those messages are never generated.

Our workaround

To restore the functionality in our deployment:

a) when a publish message has been received we added a hardcoded check whether the suffix of the topic matches "/connectionStatus" that we use, to call notify regardless the stream direction

b) added dummy subscribers listening to those messages, to force MQTT broker to generate Last Wills when needed.

[dborovcanin]

Hello, @ftylitak. This is correct - mProxy is only a TCP proxy between MQTT client and broker with some nice additional features. However, this does not seem to be the problem with mProxy because:

at the two streams of each session, only messages comming from client to the broker are notified (passed onto nats)

This is by design. We do not want to publish protocol-specific messages (such as PUBREC, say) to the internal message broker; we care only about a subset of messages. The same goes for all the messages from MQTT broker - we only tunnel them to the client.

the Last Will messages are generated from the MQTT broker only if there are active subscribers at that time. Since the mproxy is only packet forwarer, it is not a subscriber so in the general case, those messages are never generated.

Since mProxy just a is forwarder, subscriptions should not be altered (unless intentionally) and client subscriptions should result in subscriptions created on the MQTT broker - mProxy should be transparent as if there were no proxy at all from the perspective of both broker and client.

A couple of questions:
Which broker are you using at the moment?
Which version of mProxy and Magistrala are you using?
Which environment are you using for running Magistrala, MQTT broker, and mProxy?

a) when a publish message has been received we added a hardcoded check whether the suffix of the topic matches "/connectionStatus" that we use, to call notify regardless the stream direction

Also, recently, we have added support for message interceptors in mProxy, so you can inspect and alter the MQTT regardless of direction and message type (including all MQTT control packets).

[ftylitak]

Hello @dborovcanin , thank you for your response.

Indeed it is clear that this behaviour is by design.

Currently we use mainflux v0.12.0 with the docker container of mproxy of the same version. The mqtt broker is VerneMQ.
We are planning to migrate to magistrala as soon as its new version is released.
The environment that we run the services is on a single instance of Ubuntu Server 22.04 using the docker-compose.

Regarding the Last Will, I just wanted to point that out for others that might face this situation. Beacuse we were using Mainflux 0.10.0 before transitioning to 0.12.0, the mqtt implementation of mainflux at that time was different and to my understanding it was subscribing to the root topic, so it was receiving everything. This made the last will reporting functional right from the begining so we never bother investigating anything further for that.

Since magistrala evolved, it is clear that you are not oblidged to always preserve backward compatibility. Though since Last Will messages of MQTT are commonly used, it would be interesting to find out which would be the best solution to allow their messages to be written on NATS.

The message interceptors sound interesting and we will try them out.

The requirement of a pre-existing external subscriber will remain though, which is something the users need to remember.

[dborovcanin]

@ftylitak Got it. I was asking which MQTT broker are you using because NATS does not support the whole MQTT 3.1 spec, so I wanted to exclude that as a problem. It's good to have this conversation to document the behavior - and to document how mProxy is supposed to do, and what it isn't supposed to do. Instead of a current workaround, you can simply write an interceptor that checks the topic and does whatever you want. If you need some assistance, please reach out to us, we'll be more than glad to help.