Skip to content
This repository has been archived by the owner on Nov 30, 2023. It is now read-only.

Handle UnicodeDecodeError on bad UTF-8 URLs #152

Open
snarfed opened this issue Jan 20, 2020 · 1 comment
Open

Handle UnicodeDecodeError on bad UTF-8 URLs #152

snarfed opened this issue Jan 20, 2020 · 1 comment

Comments

@snarfed
Copy link

snarfed commented Jan 20, 2020

if you request a URL with bad UTF-8 escaped characters, eg /%D0%C2%BD%A8%CE%C4%BC%FE%BC%D0.rar, webapp2 crashes with a stacktrace like this:

Traceback (most recent call last):
  File "/env/lib/python3.7/site-packages/webapp2.py", line 1573, in __call__
    rv = self.handle_exception(request, response, e)
  File "/env/lib/python3.7/site-packages/webapp2.py", line 1567, in __call__
    rv = self.router.dispatch(request, response)
  File "/env/lib/python3.7/site-packages/webapp2.py", line 1301, in default_dispatcher
    route, args, kwargs = rv = self.match(request)
  File "/env/lib/python3.7/site-packages/webapp2.py", line 1241, in default_matcher
    match = route.match(request)
  File "/env/lib/python3.7/site-packages/webapp2.py", line 884, in match
    match = self.regex.match(unquote(request.path))
  File "/env/lib/python3.7/site-packages/webob/request.py", line 476, in path
    bpath = bytes_(self.path_info, self.url_encoding)
  File "/env/lib/python3.7/site-packages/webob/descriptors.py", line 70, in fget
    return req.encget(key, encattr=encattr)
  File "/env/lib/python3.7/site-packages/webob/request.py", line 165, in encget
    return bytes_(val, 'latin-1').decode(encoding)
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xd0 in position 1: invalid continuation byte

evidently the exception itself is raised by WebOb, by design. details in Pylons/webob#114. webapp2 should probably at least return HTTP 400 on bad URLs like these instead of letting WebOb's UnicodeDecodeError propagate all the way up and result in a 500.

thanks for maintaining webapp2 btw, it's great!

@snarfed
Copy link
Author

snarfed commented Jan 20, 2020

also Pylons/webob#161

Kami added a commit to StackStorm/st2 that referenced this issue Mar 13, 2021
the request URL contains invalid or incorrectly URL encoded characters.

Previously we didn't handle this scenario which means original
UnicodeDecodeError error with the stack trace got propagated to the user
which is a no go.

Related issue -
GoogleCloudPlatform/webapp2#152.
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant