Getting Invalid Response

[ossgeek314]

I'm using pynetgear in a hass.io rpi system to monitor presence by probing the device list on my Netgear Nighthawk X10 R9000 (hitting it about every 30 seconds with a query). The system works about 80% of the time and I'm able to get the list of devices without issue but for 10% of the time (and it seems random/sporadic) I get the following error:

2019-05-28 20:41:06 ERROR (SyncWorker_1) [pynetgear] Invalid response
2019-05-28 20:41:06 DEBUG (SyncWorker_1) [pynetgear] 200
{'Content-Length': '1518', 'Content-Type': 'text/xml; charset="UTF-8"', 'Server': 'Linux/2.6.15 uhttpd/1.0.0 soap/1.0'}
<?xml version="1.0" encoding="UTF-8"?>
<SOAP-ENV:Envelope
   xmlns:SOAP-ENV="http://schemas.xmlsoap.org/soap/envelope/"
   SOAP-ENV:encodingStyle="http://schemas.xmlsoap.org/soap/encoding/">
<SOAP-ENV:Body>
<m:GetAttachDeviceResponse xmlns:m="urn:NETGEAR-ROUTER:service:DeviceInfo:1">    
    <NewAttachDevice><!-- snipped for privacy --></NewAttachDevice>
</m:GetAttachDeviceResponse>
<ResponseCode>000</ResponseCo</SOAP-ENV:Body>
</SOAP-ENV:Envelope>

2019-05-28 20:41:06 ERROR (SyncWorker_1) [pynetgear] Get attached devices failed

Clearly the XML that is printed out in the debug log is invalid:

<ResponseCode>000</ResponseCo</SOAP-ENV:Body>

At first I thought it might be random dropped packets losing the response... but it is consistent in not printing the full tag. Sometimes it breaks at different spots where it only prints </Respon but it is always this closing tag that is printed.

The question is... is this actually coming from the netgear... or is this some sort of preprocessing of the response that corrupts it within pynetgear?

Either way, I see you don't actually validate the XML any way around this?

[ossgeek314]

So, gathering more information on this issue. If I run a second script on a second machine(I was running it to sniff the packets), the error does not happen on either machine. Within minutes of stopping the second script, the error returns to the original.

My gut is telling me the netgear itself is corrupting the XML somehow. I'm going to do more testing but the running theory now is 35 second polling time causes instability. I'm going to try and reduce the polling interval on the single machine and see if this causes the error to go away.

[gruijter]

Concerning instability of the router, see my post: #62 (comment)

[ossgeek314]

Setting the time interval to 15 seconds resulted in both scripts giving the error. I was able to capture the packets. The netgear itself is sending the corrupted XML

0600  65 76 69 63 65 3e 0d 0a 3c 2f 6d 3a 47 65 74 41   evice>..</m:GetA
0610  74 74 61 63 68 44 65 76 69 63 65 52 65 73 70 6f   ttachDeviceRespo
0620  6e 73 65 3e 0d 0a 3c 52 65 73 70 6f 6e 73 65 43   nse>..<ResponseC
0630  6f 64 65 3e 30 30 30 3c 2f 52 65 73 70 6f 6e 3c   ode>000</Respon<
0640  2f 53 4f 41 50 2d 45 4e 56 3a 42 6f 64 79 3e 0d   /SOAP-ENV:Body>.
0650  0a 3c 2f 53 4f 41 50 2d 45 4e 56 3a 45 6e 76 65   .</SOAP-ENV:Enve
0660  6c 6f 70 65 3e 0d 0a                              lope>..

So, I'd say this is not an issue with pynetgear (though since you aren't validating the XML but just checking for the <ResponseCode>000</ResponseCode> string... maybe a super hacky way is to just check for <ResponseCode>000</ and ignore the closing tag??)

[ctrl50]

Setting the time interval to 15 seconds resulted in both scripts giving the error. I was able to capture the packets. The netgear itself is sending the corrupted XML

0600  65 76 69 63 65 3e 0d 0a 3c 2f 6d 3a 47 65 74 41   evice>..</m:GetA
0610  74 74 61 63 68 44 65 76 69 63 65 52 65 73 70 6f   ttachDeviceRespo
0620  6e 73 65 3e 0d 0a 3c 52 65 73 70 6f 6e 73 65 43   nse>..<ResponseC
0630  6f 64 65 3e 30 30 30 3c 2f 52 65 73 70 6f 6e 3c   ode>000</Respon<
0640  2f 53 4f 41 50 2d 45 4e 56 3a 42 6f 64 79 3e 0d   /SOAP-ENV:Body>.
0650  0a 3c 2f 53 4f 41 50 2d 45 4e 56 3a 45 6e 76 65   .</SOAP-ENV:Enve
0660  6c 6f 70 65 3e 0d 0a                              lope>..

So, I'd say this is not an issue with pynetgear (though since you aren't validating the XML but just checking for the <ResponseCode>000</ResponseCode> string... maybe a super hacky way is to just check for <ResponseCode>000</ and ignore the closing tag??)

Thanks for this. i did the super hacky edit to the file and its working again. after months without it..

[ctrl50]

Reinstalled this again and same error.. Guess ill lay it out here, for anyone new. with the same issue..
re-hacking the code again:

file:

lib/python3.7/site-packages/pynetgear/__init__.py

line:465 :

("<ResponseCode>0000</ResponseCode>" in resp.text or
change to:

("<ResponseCode>0000</" in resp.text or

line:466:

"<ResponseCode>000</ResponseCode>" in resp.text))
to:
"<ResponseCode>000</" in resp.text))

Working again, HA 0.101.2

@MatMaul is this abandoned?

[VincentMasselis]

Same issue here, you're not alone..

[LintHart]

Reinstalled this again and same error.. Guess ill lay it out here, for anyone new. with the same issue..
re-hacking the code again:

file:

lib/python3.7/site-packages/pynetgear/__init__.py

line:465 :

("<ResponseCode>0000</ResponseCode>" in resp.text or
change to:

("<ResponseCode>0000</" in resp.text or

line:466:

"<ResponseCode>000</ResponseCode>" in resp.text))
to:
"<ResponseCode>000</" in resp.text))

Working again, HA 0.101.2

@MatMaul is this abandoned?

Hey @ctrl50, do you know if it’s possible to edit this in hass.io where there is seemingly no root file structure access? Running into similar problems on my D7800

Thank you, Linton

[MatMaul]

Thanks all for the detailed analysis, I have incorporated the suggested changes. Sorry for the tremendous delay, but I have been a quite more busy than usual and also don't own anymore any Netgear devices with stock firmware.