Update REST API to latest OT-BR-POSIX (TZ-217)

[FutureCow]

In the latest release the webui and REST API was added to the ESP-OTBR, but not all of the components are added to this API. Would it be possible to update the API with the latest upstream from OTBR?

https://github.com/openthread/ot-br-posix/pulls?q=is%3Apr+rest+is%3Aclosed+author%3Aagners

[chshu]

@FutureCow Thanks for reporting, we will check and add the missing APIs.

[wjbuys]

Updating to the latest REST API will enable adding an ESP thread border router to Home Assistant; see this discussion for details https://community.home-assistant.io/t/openthread-border-router-support/540715/33

[laptopfreek0]

I've done some exploratory research on this, and it appears that the missing APIs in the discussion https://community.home-assistant.io/t/openthread-border-router-support/540715/33 are not the cause for the Home Assistant rejection.

The key API mentioned in that thread is /node/dataset/active which is the same endpoint as /node/active-dataset-tlvs which according to this post https://community.home-assistant.io/t/openthread-border-router-support/540715/16 the /node/dataset/active was changed to /node/active-dataset-tlvs.
Just for fun I copied the route from /node/active-dataset-tlvs to /node/dataset/active but Home Assistant still did not find the Open Thread Border Router.

Are there any other endpoints that are not implemented that Home Assistant is looking for?

[gytxxsy]

In fact, in the latest ot-br-posix REST API, /node/active-dataset-tlvs has been replaced by /node/dataset/active. This is the related pull request: openthread/ot-br-posix#1682

It is normal for the current ESP Thread Border Router to not recognize /node/dataset/active because we are still using the old URL /node/active-dataset-tlvs. We are actively updating our esp-thread-br codes to be consistent with ot-br-posix.

If you change /node/active-dataset-tlvs in esp_br_web_base.h to /node/dataset/active, and use this URL in the browser, then you should be able to get the following data (unless BR does not has an active dataset).

However, if you only change the URL definition in esp_br_web_base.h, then data that BR actually returns is active dataset tlvs. It just a test for getting more useful information.

[laptopfreek0]

I built the latest Posix and tested the REST API /node/dataset/active it returns a JSON data structure like the following:
{ "ActiveTimestamp": { "Seconds": 1, "Ticks": 0, "Authoritative": false }, "NetworkKey": "00112233445566778899AABBCCDDEEFF", "NetworkName": "OpenThreadDemo", "ExtPanId": "1111111122222222", "MeshLocalPrefix": "fdc9:1bb7:5d18:f9ab::/64", "PanId": 4660, "Channel": 15, "PSKc": "445F2B5CA6F2A93A55CE570A70EFEECB", "SecurityPolicy": { "RotationTime": 672, "ObtainNetworkKey": true, "NativeCommissioning": true, "Routers": true, "ExternalCommissioning": true, "CommercialCommissioning": false, "AutonomousEnrollment": false, "NetworkKeyProvisioning": false, "TobleLink": true, "NonCcmRouters": false }, "ChannelMask": 134215680 }

So for testing purposes (of trying to get Home Assistant to adopt the thread border router), I added a return of the identical information. Home assistant still did not detect the Thread border router, without any logs. Maybe the pending needs to be added as well?

[oddstr13]

If you want to have a look at the information exchanged between Home Assistant and ot-br-posix, best would probably be to connect over http, and have a look at the traffic using Wireshark or tcpdump.

Alternatively, you can use ncat as a logging proxy.

[laptopfreek0]

If you want to have a look at the information exchanged between Home Assistant and ot-br-posix, best would probably be to connect over http, and have a look at the traffic using Wireshark or tcpdump.

Alternatively, you can use ncat as a logging proxy.

Awesome thanks for the stackoverflow. I was able to get tcpflow to work properly sudo tcpflow -p -i wlan0 -c port 8081 > ~/posix-logs

I got the following:

192.168.006.075.52054-192.168.006.207.08081: GET /node/dataset/active HTTP/1.1
Host: 192.168.6.207:8081
User-Agent: HomeAssistant/2023.10.5 aiohttp/3.8.5 Python/3.11
Accept: text/plain
Accept-Encoding: gzip, deflate


192.168.006.207.08081-192.168.006.075.52054: HTTP/1.1 200 OK
Access-Control-Allow-Headers: Access-Control-Allow-Headers, Origin,Accept, X-Requested-With, Content-Type, Access-Control-Request-Method, Access-Control-Request-Headers
Access-Control-Allow-Methods: DELETE, GET, OPTIONS, PUT
Access-Control-Allow-Origin: *
Connection: close
Content-Type: text/plain
Content-Length: 210

0E08000000000001000035060004001FFFE00708FDC91BB75D18F9AB0C0402A0F7F8051000112233445566778899AABBCCDDEEFF030E4F70656E54687265616444656D6F0410445F2B5CA6F2A93A55CE570A70EFEECB000300000F0208111111112222222201021234
192.168.006.075.52070-192.168.006.207.08081: GET /node/ba-id HTTP/1.1
Host: 192.168.6.207:8081
User-Agent: HomeAssistant/2023.10.5 aiohttp/3.8.5 Python/3.11
Accept: */*
Accept-Encoding: gzip, deflate


192.168.006.207.08081-192.168.006.075.52070: HTTP/1.1 200 OK
Access-Control-Allow-Headers: Access-Control-Allow-Headers, Origin,Accept, X-Requested-With, Content-Type, Access-Control-Request-Method, Access-Control-Request-Headers
Access-Control-Allow-Methods: DELETE, GET, OPTIONS, PUT
Access-Control-Allow-Origin: *
Connection: close
Content-Type: application/json
Content-Length: 34

"C2C745CB342E150B5C251048237562BA"
192.168.006.075.52084-192.168.006.207.08081: GET /node/dataset/active HTTP/1.1
Host: 192.168.6.207:8081
User-Agent: HomeAssistant/2023.10.5 aiohttp/3.8.5 Python/3.11
Accept: text/plain
Accept-Encoding: gzip, deflate


192.168.006.207.08081-192.168.006.075.52084: HTTP/1.1 200 OK
Access-Control-Allow-Headers: Access-Control-Allow-Headers, Origin,Accept, X-Requested-With, Content-Type, Access-Control-Request-Method, Access-Control-Request-Headers
Access-Control-Allow-Methods: DELETE, GET, OPTIONS, PUT
Access-Control-Allow-Origin: *
Connection: close
Content-Type: text/plain
Content-Length: 210

0E08000000000001000035060004001FFFE00708FDC91BB75D18F9AB0C0402A0F7F8051000112233445566778899AABBCCDDEEFF030E4F70656E54687265616444656D6F0410445F2B5CA6F2A93A55CE570A70EFEECB000300000F0208111111112222222201021234 

I'll keep digging, but it does look like the issue is probably the allowed methods of DELETE, GET, Options, And Put

[laptopfreek0]

Finally! While I've had to hack and hardcode the string currently, because I'm not great at C I got the thread border router added to homeassistant.

The key is that the /node/dataset/active needs to be in plain/text and there cannot be any quotes. Just using the existing active dataset tlvs runs it through JSON which adds quotes at the beginning of the string, and the current implementation only returns JSON.

[laptopfreek0]

PR for the specific /node/dataset/active being returned in plain/text for homeassistant adoption.

[chshu]

@FutureCow @laptopfreek0 The active/pending dataset REST APIs have been added in this commit: 0b08a3a.

The current supported REST APIs are listed here: https://docs.espressif.com/projects/esp-thread-br/en/latest/codelab/web-gui.html#thread-rest-apis

Thanks for your support, let us know if any further question or issues.