Apprise sends strange Signs

[kickbutts]

Hi there,
i tried to use mqtt to send my detections to my smartHome.
I am using this Notification body:
{"Common_Name": "$comname", "Scientific_Name": "$sciname", "Confidence_Score": "$confidence"}
Sending this via mqtt, this is what arrives at my mqtt server:

\{"Common_Name": "Dohle", "Scientific_Name": "Corvus monedula", "Confidence_Score": "0.8049797"\}

When trying to parse this to json i am getting this error message: "Unexpected token \ in JSON at position 0"
The problem is \, which i did not add to the message.
Sending {"Common_Name": "$comname", "Scientific_Name": "$sciname", "Confidence_Score": "$confidence"} via mqtt works fine.
Any ideas?
Thanks!

[deepcoder]

I believe I encountered similar issues. 1st the text that comes out of the recognition engine does some interesting 'hoops' to deal with embedded single quote marks. Dealing with quote marks in csv type data structures is always a challenge. 2nd, I believe I found that Apprise does not create valid JSON all the time. I ended up writing my own MQTT publisher and not using Apprise. The code and setup is in the link below, perhaps it will give you some ideas.

https://gist.github.com/deepcoder/c309087c456fc733435b47d83f4113ff

[caronc]

As the developer of Apprise, i'd love to fix the issue, but no one reported the bug. I just stumbled upon this by accident.

Can you show me specifically what you're doing? MQTT is just a message broker, it's actually up to the system passing in the data to format it how they want it. Apprise does nothing but offer a common notification proxy to multiple platforms (MQTT included). If you pass in X to Apprise, it passes it along just like that. If you pass in mangled JSON, it passes it along just like that. Some might push XML, some might push YAML. Apprise just passes along what it gets (as-is).

But if you can see a use-case where Apprise should assume the data passed to it will be obstructed or bad and should vet/clean it up, i can try to see what i can do here. This wouldn't be an ideal approach though i don't think. If i could see a code example where it is failing for you guys, maybe i can help you debug it as well.

[deepcoder]

When I was debugging what was coming out of BirdNET and into Apprise and then what Apprise was sending out to MQTT, unfortunately I did not document the issues or whether they seemed to be caused by 'usage' or by issue in Apprise.

Three things I remember from my work via the original post above:

1. the ellipse left or right was being sent to MQTT with what appeared to be a leading backslash. I guess trying to do some type of 'escaping' of the ellipse.

2. In JSON, I understand that the double quote is the only type of quote used in the JSON language definition. I do not remember the details, however I believe BirdNET was using single quotes by default, but then when the bird name in the string contained a single quote, for example shown below in the case of Cooper's Hawk, flipped to using double quotes. As I said, I did not document the details, but the resulting 'JSON' that went to MQTT was not valid. If I has a penny for every programming hour that was spend frustrating over dealing with the battle of quotes between programming languages and human languages ;-)

3. BirdNET's usage of Apprise to send to MQTT published a 'hybrid' message of non-JSON and JSON. The receiver on the other end is going have a real tough time decoding a message like that, as MQTT just presents what it gets without any template, it's either JSON or text. I think was caused by BirdNET passing a 'title' and 'body', title was text, body was JSON.

The utility 'jq' is a good test to see if you have or do not have valid JSON.

Good hunting!

16:12:07---3.0;6.0-('Turdus pallidus_Pale Thrush', 0.058778297)
16:12:07---6.0;9.0-("Accipiter cooperii_Cooper's Hawk", 0.038338106)

[kickbutts]

Thanks for your answers.
I tested a bit and look at some code and i am pretty sure the Problem is not Apprise (@caronc ). I think (as @deepcoder ) mentioned, BirdnetPi inserts the \ not Apprise. So this might bee the place to look.
I solved my problem by editing the Sting which is send by Apprise via MQTT. I wrote a function to replace the \ in the message, before i parsed it to json. This works for me and might help others.
@caronc As Apprise just send the data which is feed into it, i dont think it is a think for you to change. It might be possible to check if the message body is for example vaild json. But as i understood Apprise, this is a task which should be done before feeding the message into Apprise.

[louispires]

I wrote a function to replace the \ in the message, before i parsed it to json. This works for me and might help others.

Could you kindly share this function?

[kickbutts]

I am not sure this will help you, but here it is:

var text = msg.payload; msg.payload = text.replace(/\\/g,''); return msg;

I is a NodeRed Function written in JavaScript.