Problem with encoded urls (%3A)

[mattiaswal]

When sending a RESTCONF request using python requests , it is automatically URL encode as percent-encoded but after 96cbf73 I only get:

# requests.exceptions.HTTPError: 400 Client Error: Bad Request for url: https://[fe80::2a0:85ff:fe00:301%d3a]:443/restconf/ds/ietf-datastores%3Aoperational/ietf-yang-library%3Amodules-state
# 12:44:18 infamy0:test # 

If i remove that patch, it works again.

 curl -v -u admin:admin -H "Accept: application/yang-data+json" -kX GET "https://[fe80::2a0:85ff:fe00:301%d3a]:443/restconf/ds/ietf-dat
astores:operational/ietf-yang-library:modules-state"

works though.

[jktjkt]

I'm afraid that our code is correct, but if you can point out to a section in any relevant RFC which proves us wrong, we'll of course fix our bug(s).

The problem is that the RESTCONF RFC defines rules on what MUST be percent-encoded, and if these rules are not followed, you will have problems working with list and leaf-list string keys. This is also nicely illustrated in a Stackoverflow post. But let's look into the standards:

• RESTCONF defers to RFC 3986 for a definition of URL and its processing
• the percent-encoding definition in RFC 3986 says that it MUST NOT be used for some reserved characters
• it also explicitly says that two URL strings which are only different in percent-encoding of a reserved character, are in fact two different URLs
• the colon : is listed as one of the reserved characters.

So, you could argue that our implementation is broken because it does not accept an URL like this:

/restconf/ds/ietf-datastores:operationa%6C

...because that one should be interpreted as /restconf/ds/ietf-datastores:operational, but I think it would be actually a violation of 3986 if we treated this:

/restconf/ds/ietf-datastores%3Aoperational

...as an URL that's equivalent to /restconf/ds/ietf-datastores:operational.

I think that urllib3/urllib3#3135 is the relevant bugreport for the Python implementation.

But again, this is a complex matter, so if you can point out any standard which says otherwise, please say so.