Data corrupted between MQTT and Jee Interfacers

[delboy711]

I am sending transmit data via MQTT to a remote node. The data is a single value sent to Node 24.
The Jee interfacer passes the message apparently OK, but it gets transmitted to the wrong node ID, and the message is sent as 3 bytes instead of 2.

Log sequence is
MQTT Nodeid: 24 values: 1526
MQTT 9 Sent to channel' : ToRFM12
RFM2Pi 9 sent TX packet: 24,246,5,s
acknowledged command: > 24,246,5,0s
confirmed sent packet size: -> 3 b

The packet arrives at Node 5

The stanza in emonhub.conf for the node is

[[24]]
nodename = solar_EVSE
firmware =solar_EVSE
hardware = solar_EVSE
[[[tx]]]
names = dacsetpoint
units = C
scales = 1
datacode = h

I can get it working by patching EmonHubJeeInterfacer.py send() function
` def send (self, cargo):

    f = cargo
    cmd = "s"

    if self.getName() in f.encoded:
        data = f.encoded[self.getName()]
    else:
        data = f.realdata

    payload = ""
    for value in range(1, len(data)):
        if int(data[value]) < 0 or int(data[value]) > 255:
            self._log.warning(self.name + " discarding Tx packet: values out of scope" )
            return
        payload += str(int(data[value]))+","

    payload += data[0] + cmd

    self._log.debug(str(f.uri) + " sent TX packet: " + payload)
    self._ser.write(payload)

`

The log sequence then looks like
MQTT Nodeid: 24 values: 1526
MQTT 23 Sent to channel' : ToRFM12
RFM2Pi 23 sent TX packet: 246,5,24s
RFM2Pi acknowledged command: > 246,5,24s
RFM2Pi confirmed sent packet size: -> 2 b

The node receives that OK

It is not clear to me if it is the MQTT Interfacer malforming the command, or the Jee Interfacer misreading it. If the latter, then here is the fix:-)

Thanks

Derek

[alandpearson]

Howdy folks,
Me again... I recently upgraded my 2016 SD card to the latest version (2021) and noticed sending had broken.
I'm no python guy, but in debugging noticed this was down to the python2->3 transition.

Thanks Derek for helping me solve the last part of the puzzle here.

First, lets clarify how devices in the OEM infrastructure format packets they send and expect to receive.
When sending, the first byte is always the DEST address and it's up to the RECEIVER to check this byte to see if they are interested in this packet.
All packets are sent as 'broadcasts' (when using the serial interface/ RFDemo they end in 0s) so all nodes will receive them in the software stack. The receiver checks the RF12_HDR and can see who sent the packet if they so desire. That's how it works in OEM devices (sniff the airwaves using the RF12Demo and you will see). This is due to a limitation in JeeLib where you can only specify the TX or RX id, but not both unless you encode in the data you send.

With that knowledge, the patch above isn't quite correct (in short it needs to start at data[0] which is the DEST address as intended - which Derek mentions is an extra byte, it's not... that's how OEM packets are formatted) and then needs to append 0s to the end of the payload which will be the serial string. @delboy711 - for your application you need to ensure you account for the extra 'first' byte which is the DEST address as described above.

Correct code that works for emonHub 2021 is below
File: /opt/openenergymonitor/emonhub/src/interfacers/EmonHubJeeInterfacer.py

def send(self, cargo):
    f = cargo
    cmd = "s"

    if self.getName() in f.encoded:
        data = f.encoded[self.getName()]
    else:
        data = f.realdata

    payload = ""
    for value in range(0, len(data)):
        if int(data[value]) < 0 or int(data[value]) > 255:
            self._log.warning(self.name + " discarding Tx packet: values out of scope" )
            return
        payload += str(int(data[value]))+","

    payload += '0' + cmd

    self._log.info(str(f.uri) + " sent TX packet: " + payload)
    self._ser.write(payload.encode())

You will also need to make a one line change to EmonHubMqttInterfacer.py (line 263 - append .decode to payload = msg.payload)

        nodeid = int(topic_parts[2])
        payload = msg.payload.decode()
        realdata = payload.split(",")
        self._log.debug("Nodeid: %s values: %s", nodeid, msg.payload)

Hope this helps someone, it certainly got my emonGLCD and other toys that rely on emonHub for RF sending working again.

@glynhudson I'll submit a patch soon

[delboy711]

This is a blast from the past! Thanks for getting to the bottom of this one Alan.
Now I need to hunt out the source for my application.

[alandpearson]

Fixed in main codebase, pull request submitted :
#175