MQTT-C endless loop if sendbuffer is too large

[Bauteiltoeter]

After enlarging the receive / sendBuffer for MQTT-C to 16MB my application started to go into endless loops at 100% CPU time after some time.

My application is sending and receiveing QoS0-Messages only.

Investigating this issue showed that the do-while-loop inside of __mqtt_next_pid was spinning endlessly because it couldn't find any unused pids. Analysing client->mq showed that there where 65535 packages in the queue.

I tried to find the mechanism where this queue is cleared. The only way that I could find to clear the queue is the callstack
mqtt_publish() -> MQTT_CLIENT_TRY_PACK() -> mqtt_pack_publish_request() -> mqtt_pack_fixed_header() returning 0 because there is not enough space in the transmit buffer.

This means there is no "normal" way this queue is cleared, it's only cleared if it's full. If more than 65535 packages fit into the queue it will fail.

My application has a lot of small messages mixed with very big messages, this will make the appearance of the bug much more likely.

Steps to reproduce:

• Start with the examples/simple_publisher.c
• Change sendbuf[] size to 0xFFFFFF (16MB)
• Change the while()-Loop to a while(1) to make it endless
• add a usleep(5000u) to slow it down a little bit
• Build, start and grab a coffee
• CPU usage will slowly increase until it will hit a 100% wall after the 65535th package.