Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Provide sensible errors on network issues #385

Draft
wants to merge 1 commit into
base: master
Choose a base branch
from

Conversation

dagwieers
Copy link
Collaborator

@dagwieers dagwieers commented Jul 15, 2019

Currently the add-on works fine if all is well, however various things can go wrong which makes the add-on bail out with no proper error.

And since the cause of network errors could be anywhere, we do not want the end-user to have to dig deep into Kodi logs to find probable suspects.

So we do want to return the exact error message (i.e. Connection refused, No route to host, etc.) so the user can relate it to other possible problems. (WIFI issues, Internet down or unreliable, etc.)

This PR relies on #384 and is far from finished, there are many places in the code that may be affected by network issues. So we have to make sure we are doing this as efficient as possible so the code remains readable and transparent.

@dagwieers dagwieers added the enhancement New feature or request label Jul 15, 2019
@dagwieers dagwieers added this to the Future milestone Jul 15, 2019
@dagwieers dagwieers changed the title Attempt to provide sensible errors on network issues Provide sensible errors on network issues Jul 15, 2019
@dagwieers dagwieers force-pushed the error_handling branch 2 times, most recently from 1d84eae to cdcc95c Compare July 15, 2019 16:28
@dagwieers
Copy link
Collaborator Author

Part of me would like to put the actual error message (e.g. Connection Refused) into the OK dialog heading. But I don't know (yet) if the error message is always short and concise.

@mediaminister
Copy link
Collaborator

I don't think its a good idea to put SSL: CERTIFICATE_VERIFY_FAILED or An existing connection was forcibly closed by the remote host in a dialog heading

@dagwieers
Copy link
Collaborator Author

BTW I haven't been testing this yet. I just started adding it to a part that failed for me once, and then added it to similar calls as well. But I have not induced any errors to see what gets caught. Most likely we want to add an abstraction layer for this if we decided we want/need this after sufficient testing.

@dagwieers
Copy link
Collaborator Author

dagwieers commented Sep 2, 2019

[Errno 10013] An attempt was made to access a socket in a way forbidden by its access permissions

An issue I encountered on Windows 10 using Kodi 18.4 when the corporate VPN was enabled, and my local proxy was still enabled for testing.

2019-09-02 13:55:30.333 T:16580   ERROR: EXCEPTION Thrown (PythonToCppException) : -->Python callback/script returned the following error<--
                                             - NOTE: IGNORING THIS CAN LEAD TO MEMORY LEAKS!
                                            Error Type: <class 'urllib2.URLError'>
                                            Error Contents: <urlopen error [Errno 10013] An attempt was made to access a socket in a way forbidden by its access permissions>
                                            Traceback (most recent call last):
                                              File "C:\Users\dwieers\AppData\Roaming\Kodi\addons\plugin.video.vrt.nu\resources\lib\addon_plugin.py", line 8, in <module>
                                                run(sys.argv)
                                              File "C:\Users\dwieers\AppData\Roaming\Kodi\addons\plugin.video.vrt.nu\resources\lib\addon.py", line 254, in run
                                                plugin.run(argv)
                                              File "C:\Users\dwieers\AppData\Roaming\Kodi\addons\script.module.routing\lib\routing.py", line 115, in run
                                                self._dispatch(path)
                                              File "C:\Users\dwieers\AppData\Roaming\Kodi\addons\script.module.routing\lib\routing.py", line 126, in _dispatch
                                                view_func(**kwargs)
                                              File "C:\Users\dwieers\AppData\Roaming\Kodi\addons\plugin.video.vrt.nu\resources\lib\addon.py", line 24, in main_menu
                                                VRTPlayer(kodi).show_main_menu()
                                              File "C:\Users\dwieers\AppData\Roaming\Kodi\addons\plugin.video.vrt.nu\resources\lib\vrtplayer.py", line 22, in show_main_menu
                                                self._favorites.get_favorites(ttl=60 * 60)
                                              File "C:\Users\dwieers\AppData\Roaming\Kodi\addons\plugin.video.vrt.nu\resources\lib\favorites.py", line 36, in get_favorites
                                                xvrttoken = TokenResolver(self._kodi).get_xvrttoken(token_variant='user')
                                              File "C:\Users\dwieers\AppData\Roaming\Kodi\addons\plugin.video.vrt.nu\resources\lib\tokenresolver.py", line 76, in get_xvrttoken
                                                token = self._get_fresh_token(refresh_token, 'X-VRT-Token', token_variant=token_variant)
                                              File "C:\Users\dwieers\AppData\Roaming\Kodi\addons\plugin.video.vrt.nu\resources\lib\tokenresolver.py", line 253, in _get_fresh_token
                                                opener.open(req)
                                              File "C:\Program Files\Kodi\system\python\Lib\urllib2.py", line 429, in open
                                                response = self._open(req, data)
                                              File "C:\Program Files\Kodi\system\python\Lib\urllib2.py", line 447, in _open
                                                '_open', req)
                                              File "C:\Program Files\Kodi\system\python\Lib\urllib2.py", line 407, in _call_chain
                                                result = func(*args)
                                              File "C:\Program Files\Kodi\system\python\Lib\urllib2.py", line 1241, in https_open
                                                context=self._context)
                                              File "C:\Program Files\Kodi\system\python\Lib\urllib2.py", line 1198, in do_open
                                                raise URLError(err)
                                            URLError: <urlopen error [Errno 10013] An attempt was made to access a socket in a way forbidden by its access permissions>
                                            -->End of Python script error report<--
2019-09-02 13:55:30.351 T:11104   ERROR: XFILE::CDirectory::GetDirectory - Error getting plugin://plugin.video.vrt.nu/
2019-09-02 13:55:30.361 T:15392   ERROR: CGUIMediaWindow::GetDirectory(plugin://plugin.video.vrt.nu/) failed

@dagwieers
Copy link
Collaborator Author

Just as a reference, here is a retry-mechanism we could implement as well:
https://stackoverflow.com/questions/9446387/how-to-retry-urllib2-request-when-fails

@dagwieers
Copy link
Collaborator Author

dagwieers commented Oct 24, 2019

HTTPError: HTTP Error 500: Internal Server Error

Here is another issue from an unexpected HTTP Error 500 that causes a backtrace:

2019-10-24 22:56:53.028 T:1307829104  NOTICE: [plugin.video.vrt.nu] Got cached token '/storage/.kodi/userdata/addon_data/plugin.video.vrt.nu/tokens/vrtloginrt.tkn'
2019-10-24 22:56:53.031 T:1307829104  NOTICE: [plugin.video.vrt.nu] URL get: https://token.vrt.be/refreshtoken
2019-10-24 22:56:53.338 T:1307829104  NOTICE: [plugin.video.vrt.nu] URL post: https://media-services-public.vrt.be/vualto-video-aggregator-web/rest/external/v1/tokens
2019-10-24 22:56:53.847 T:1307829104   ERROR: EXCEPTION Thrown (PythonToCppException) : -->Python callback/script returned the following error<--
                                             - NOTE: IGNORING THIS CAN LEAD TO MEMORY LEAKS!
                                            Error Type: <class 'urllib2.HTTPError'>
                                            Error Contents: HTTP Error 500: Internal Server Error
                                            Traceback (most recent call last):
                                              File "/storage/.kodi/addons/plugin.video.vrt.nu/resources/lib/addon_plugin.py", line 8, in <module>
                                                run(sys.argv)
                                              File "/storage/.kodi/addons/plugin.video.vrt.nu/resources/lib/addon.py", line 300, in run
                                                plugin.run(argv)
                                              File "/storage/.kodi/addons/script.module.routing/lib/routing.py", line 130, in run
                                                self._dispatch(self.path)
                                              File "/storage/.kodi/addons/script.module.routing/lib/routing.py", line 141, in _dispatch
                                                view_func(**kwargs)
                                              File "/storage/.kodi/addons/plugin.video.vrt.nu/resources/lib/addon.py", line 265, in play_id
                                                VRTPlayer(kodi).play(dict(video_id=video_id, publication_id=publication_id))
                                              File "/storage/.kodi/addons/plugin.video.vrt.nu/resources/lib/vrtplayer.py", line 350, in play
                                                stream = _streamservice.get_stream(video)
                                              File "/storage/.kodi/addons/plugin.video.vrt.nu/resources/lib/streamservice.py", line 195, in get_stream
                                                stream_json = self._get_stream_json(api_data, roaming)
                                              File "/storage/.kodi/addons/plugin.video.vrt.nu/resources/lib/streamservice.py", line 152, in _get_stream_json
                                                playertoken = self._tokenresolver.get_playertoken(token_url, token_variant='ondemand', roaming=roaming)
                                              File "/storage/.kodi/addons/plugin.video.vrt.nu/resources/lib/tokenresolver.py", line 65, in get_playertoken
                                                token = self._get_new_playertoken(token_url, headers, token_variant)
                                              File "/storage/.kodi/addons/plugin.video.vrt.nu/resources/lib/tokenresolver.py", line 126, in _get_new_playertoken
                                                playertoken = json.load(urlopen(req))
                                              File "/usr/lib/python2.7/urllib2.py", line 154, in urlopen
                                              File "/usr/lib/python2.7/urllib2.py", line 435, in open
                                              File "/usr/lib/python2.7/urllib2.py", line 548, in http_response
                                              File "/usr/lib/python2.7/urllib2.py", line 473, in error
                                              File "/usr/lib/python2.7/urllib2.py", line 407, in _call_chain
                                              File "/usr/lib/python2.7/urllib2.py", line 556, in http_error_default
                                            HTTPError: HTTP Error 500: Internal Server Error
                                            -->End of Python script error report<--
2019-10-24 22:56:54.129 T:1936899616   ERROR: Playlist Player: skipping unplayable item: 0, path [plugin://plugin.video.vrt.nu/play/id/vid-92aa462e-9f00-47da-825a-f3ac50e62d50/pbs-pub-85bf15d5-3990-47bb-a9e7-7798ad8f447c]

While we cannot fix this, we ought to make sure this gives a proper error and possibly a cause/remediation text.

Update: In this case, I tried multiple times and restarted the add-on, which didn't work. A restart of Kodi did work. So I guess invalidating tokens did help. But it is also clear that it should not be playing the file if we have an issue with getting a new token. cc @mediaminister

@dagwieers
Copy link
Collaborator Author

dagwieers commented Oct 25, 2019

Tunnel connection failed: 503 Service Unavailable

When Telenet was doing maintenance on my Internet connection, my video froze.
After some time it returned back to the menu listing, and refreshed the container.
Which failed as below (when using a local proxy).

2019-10-25 05:04:35.450 T:1173316480  NOTICE: [plugin.video.vrt.nu] URL get: https://vrtnu-api.vrt.be/search?i=video&facets[programUrl]=[]&from=1&size=50
2019-10-25 05:04:35.569 T:1173316480   ERROR: EXCEPTION Thrown (PythonToCppException) : -->Python callback/script returned the following error<--
                                             - NOTE: IGNORING THIS CAN LEAD TO MEMORY LEAKS!
                                            Error Type: <class 'urllib2.URLError'>
                                            Error Contents: <urlopen error Tunnel connection failed: 503 Service Unavailable>
                                            Traceback (most recent call last):
                                              File "/storage/.kodi/addons/plugin.video.vrt.nu/resources/lib/addon_plugin.py", line 8, in <module>
                                                run(sys.argv)
                                              File "/storage/.kodi/addons/plugin.video.vrt.nu/resources/lib/addon.py", line 300, in run
                                                plugin.run(argv)
                                              File "/storage/.kodi/addons/script.module.routing/lib/routing.py", line 130, in run
                                                self._dispatch(self.path)
                                              File "/storage/.kodi/addons/script.module.routing/lib/routing.py", line 141, in _dispatch
                                                view_func(**kwargs)
                                              File "/storage/.kodi/addons/plugin.video.vrt.nu/resources/lib/addon.py", line 97, in favorites_recent
                                                VRTPlayer(kodi).show_recent_menu(page=page, use_favorites=True)
                                              File "/storage/.kodi/addons/plugin.video.vrt.nu/resources/lib/vrtplayer.py", line 246, in show_recent_menu
                                                episode_items, sort, ascending, content = self._apihelper.list_episodes(page=page, use_favorites=use_favorites, variety='recent')
                                              File "/storage/.kodi/addons/plugin.video.vrt.nu/resources/lib/apihelper.py", line 122, in list_episodes
                                                page=page, use_favorites=use_favorites, variety=variety, cache_file=cache_file)
                                              File "/storage/.kodi/addons/plugin.video.vrt.nu/resources/lib/apihelper.py", line 556, in get_episodes
                                                search_json = json.load(urlopen(req))
                                              File "/usr/lib/python2.7/urllib2.py", line 154, in urlopen
                                              File "/usr/lib/python2.7/urllib2.py", line 429, in open
                                              File "/usr/lib/python2.7/urllib2.py", line 447, in _open
                                              File "/usr/lib/python2.7/urllib2.py", line 407, in _call_chain
                                              File "/usr/lib/python2.7/urllib2.py", line 1241, in https_open
                                              File "/usr/lib/python2.7/urllib2.py", line 1198, in do_open
                                            URLError: <urlopen error Tunnel connection failed: 503 Service Unavailable>
                                            -->End of Python script error report<--
2019-10-25 05:04:35.900 T:1722925952   ERROR: GetDirectory - Error getting plugin://plugin.video.vrt.nu/favorites/recent
2019-10-25 05:04:35.904 T:1937488320   ERROR: CGUIMediaWindow::GetDirectory(plugin://plugin.video.vrt.nu/favorites/recent) failed

After this error, I got transfered to the default Videos menu.

When I then opened VRT NU, the main menu opened fine, but going to My favorites, showed the "No followed programs found" dialog.
As if favorites listing was empty (so it was not using my cache at all).

Doing this again did open the favorites menu, but opening My programs (without proxy) would result again in:

2019-10-25 05:09:56.606 T:1213191040  NOTICE: [plugin.video.vrt.nu] URL get: https://vrtnu-api.vrt.be/search?i=video&facets[programType]=oneoff&size=300
2019-10-25 05:09:56.758 T:1213191040   ERROR: EXCEPTION Thrown (PythonToCppException) : -->Python callback/script returned the following error<--
                                             - NOTE: IGNORING THIS CAN LEAD TO MEMORY LEAKS!
                                            Error Type: <class 'urllib2.URLError'>
                                            Error Contents: <urlopen error Tunnel connection failed: 503 Service Unavailable>
                                            Traceback (most recent call last):
                                              File "/storage/.kodi/addons/plugin.video.vrt.nu/resources/lib/addon_plugin.py", line 8, in <module>
                                                run(sys.argv)
                                              File "/storage/.kodi/addons/plugin.video.vrt.nu/resources/lib/addon.py", line 300, in run
                                                plugin.run(argv)
                                              File "/storage/.kodi/addons/script.module.routing/lib/routing.py", line 130, in run
                                                self._dispatch(self.path)
                                              File "/storage/.kodi/addons/script.module.routing/lib/routing.py", line 141, in _dispatch
                                                view_func(**kwargs)
                                              File "/storage/.kodi/addons/plugin.video.vrt.nu/resources/lib/addon.py", line 82, in favorites_programs
                                                VRTPlayer(kodi).show_tvshow_menu(use_favorites=True)
                                              File "/storage/.kodi/addons/plugin.video.vrt.nu/resources/lib/vrtplayer.py", line 179, in show_tvshow_menu
                                                tvshow_items = self._apihelper.list_tvshows(use_favorites=use_favorites)
                                              File "/storage/.kodi/addons/plugin.video.vrt.nu/resources/lib/apihelper.py", line 81, in list_tvshows
                                                oneoffs = self.get_episodes(variety='oneoff', cache_file=cache_file)
                                              File "/storage/.kodi/addons/plugin.video.vrt.nu/resources/lib/apihelper.py", line 556, in get_episodes
                                                search_json = json.load(urlopen(req))
                                              File "/usr/lib/python2.7/urllib2.py", line 154, in urlopen
                                              File "/usr/lib/python2.7/urllib2.py", line 429, in open
                                              File "/usr/lib/python2.7/urllib2.py", line 447, in _open
                                              File "/usr/lib/python2.7/urllib2.py", line 407, in _call_chain
                                              File "/usr/lib/python2.7/urllib2.py", line 1241, in https_open
                                              File "/usr/lib/python2.7/urllib2.py", line 1198, in do_open
                                            URLError: <urlopen error Tunnel connection failed: 503 Service Unavailable>
                                            -->End of Python script error report<--

And then my Internet connection was back.

@dagwieers
Copy link
Collaborator Author

dagwieers commented Oct 27, 2019

No JSON object could be decoded

Another error occurred out of the blue.
See #554

2019-10-27 23:37:21.071 T:1535468416  NOTICE: [plugin.video.vrt.nu] URL get: https://token.vrt.be/refreshtoken
2019-10-27 23:37:21.426 T:1364509568  NOTICE: [plugin.video.vrt.nu] Access: plugin://plugin.video.vrt.nu/favorites/recent
2019-10-27 23:37:21.589 T:1345446784  NOTICE: [plugin.video.vrt.nu] URL post: https://video-user-data.vrt.be/favorites
2019-10-27 23:37:21.632 T:1535468416  NOTICE: [plugin.video.vrt.nu] URL post: https://video-user-data.vrt.be/resume_points/contentdamvrt20191024dezevendedagr2019a0027depotwp00149014
2019-10-27 23:37:21.632 T:1535468416  NOTICE: [plugin.video.vrt.nu] URL post data:: {"url": "/vrtnu/a-z/de-zevende-dag/2019/de-zevende-dag-d20191027/", "position": 1267.3970947265625, "total": 7161.00048828125, "watchLater": false}
2019-10-27 23:37:21.684 T:1364509568   ERROR: EXCEPTION Thrown (PythonToCppException) : -->Python callback/script returned the following error<--
                                             - NOTE: IGNORING THIS CAN LEAD TO MEMORY LEAKS!
                                            Error Type: <type 'exceptions.ValueError'>
                                            Error Contents: No JSON object could be decoded
                                            Traceback (most recent call last):
                                              File "/storage/.kodi/addons/plugin.video.vrt.nu/resources/lib/addon_plugin.py", line 8, in <module>
                                                run(sys.argv)
                                              File "/storage/.kodi/addons/plugin.video.vrt.nu/resources/lib/addon.py", line 300, in run
                                                plugin.run(argv)
                                              File "/storage/.kodi/addons/script.module.routing/lib/routing.py", line 130, in run
                                                self._dispatch(self.path)
                                              File "/storage/.kodi/addons/script.module.routing/lib/routing.py", line 141, in _dispatch
                                                view_func(**kwargs)
                                              File "/storage/.kodi/addons/plugin.video.vrt.nu/resources/lib/addon.py", line 97, in favorites_recent
                                                VRTPlayer(kodi).show_recent_menu(page=page, use_favorites=True)
                                              File "/storage/.kodi/addons/plugin.video.vrt.nu/resources/lib/vrtplayer.py", line 243, in show_recent_menu
                                                self._favorites.refresh(ttl=5 * 60 if use_favorites else 60 * 60)
                                              File "/storage/.kodi/addons/plugin.video.vrt.nu/resources/lib/favorites.py", line 37, in refresh
                                                xvrttoken = TokenResolver(self._kodi).get_xvrttoken(token_variant='user')
                                              File "/storage/.kodi/addons/plugin.video.vrt.nu/resources/lib/tokenresolver.py", line 71, in get_xvrttoken
                                                token = self._get_cached_token('X-VRT-Token', token_variant)
                                              File "/storage/.kodi/addons/plugin.video.vrt.nu/resources/lib/tokenresolver.py", line 100, in _get_cached_token
                                                token = json.load(fdesc)
                                              File "/usr/lib/python2.7/json/__init__.py", line 291, in load
                                              File "/usr/lib/python2.7/json/__init__.py", line 339, in loads
                                              File "/usr/lib/python2.7/json/decoder.py", line 364, in decode
                                              File "/usr/lib/python2.7/json/decoder.py", line 382, in raw_decode
                                            ValueError: No JSON object could be decoded
                                            -->End of Python script error report<--
2019-10-27 23:37:21.961 T:1527075712   ERROR: GetDirectory - Error getting plugin://plugin.video.vrt.nu/favorites/recent
2019-10-27 23:37:21.972 T:1936894400   ERROR: CGUIMediaWindow::GetDirectory(plugin://plugin.video.vrt.nu/favorites/recent) failed
2019-10-27 23:37:22.390 T:1345446784  NOTICE: [plugin.video.vrt.nu] Write cache 'favorites.json'.
2019-10-27 23:37:22.765 T:1345446784  NOTICE: [plugin.video.vrt.nu] Cache 'resume_points.json' is fresh, expires in 4 minutes and 59 seconds.
2019-10-27 23:37:22.770 T:1345446784  NOTICE: [plugin.video.vrt.nu] Cache 'my-recent-1.json' is fresh, expires in 14 minutes and 42 seconds.

So the assumption is that we get (from time to time?) empty responses and we may cache these responses, where we ought to escalate/log them instead. See #554 #556 #558

As a means to get a more reliable add-on and improve reporting we should always check/catch JSON issues and report/log them.

@dagwieers
Copy link
Collaborator Author

dagwieers commented Nov 18, 2019

urllib.error.HTTPError: HTTP Error 504: Gateway Time-out

This happened during integration tests on Travis CI:
https://travis-ci.org/dagwieers/plugin.video.vrt.nu/jobs/617011574

VRT NU infrastructure seems to be having issues currently, resulting in:

======================================================================
ERROR: test_follow_unfollow (test.test_favorites.TestFavorites)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/travis/build/pietje666/plugin.video.vrt.nu/test/test_favorites.py", line 93, in test_follow_unfollow
    self._favorites.follow(program=program, title=program_title)
  File "/home/travis/build/pietje666/plugin.video.vrt.nu/resources/lib/favorites.py", line 108, in follow
    succeeded = self.update(program, title, True)
  File "/home/travis/build/pietje666/plugin.video.vrt.nu/resources/lib/favorites.py", line 87, in update
    result = urlopen(req)
  File "/opt/python/3.6.7/lib/python3.6/urllib/request.py", line 223, in urlopen
    return opener.open(url, data, timeout)
  File "/opt/python/3.6.7/lib/python3.6/urllib/request.py", line 532, in open
    response = meth(req, response)
  File "/opt/python/3.6.7/lib/python3.6/urllib/request.py", line 642, in http_response
    'http', request, response, code, msg, hdrs)
  File "/opt/python/3.6.7/lib/python3.6/urllib/request.py", line 570, in error
    return self._call_chain(*args)
  File "/opt/python/3.6.7/lib/python3.6/urllib/request.py", line 504, in _call_chain
    result = func(*args)
  File "/opt/python/3.6.7/lib/python3.6/urllib/request.py", line 650, in http_error_default
    raise HTTPError(req.full_url, code, msg, hdrs, fp)
urllib.error.HTTPError: HTTP Error 504: Gateway Time-out

And also this occurred:

======================================================================
ERROR: test_play_livestream_by_url_route (test.test_routing.TestRouting)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/travis/build/dagwieers/plugin.video.vrt.nu/test/test_routing.py", line 220, in test_play_livestream_by_url_route
    addon.run(['plugin://plugin.video.vrt.nu/play/url/https://www.vrt.be/vrtnu/kanalen/canvas/', '0', ''])
  File "/home/travis/build/dagwieers/plugin.video.vrt.nu/resources/lib/addon.py", line 299, in run
    plugin.run(argv)
  File "/home/travis/virtualenv/python3.6.7/lib/python3.6/site-packages/routing.py", line 130, in run
    self._dispatch(self.path)
  File "/home/travis/virtualenv/python3.6.7/lib/python3.6/site-packages/routing.py", line 141, in _dispatch
    view_func(**kwargs)
  File "/home/travis/build/dagwieers/plugin.video.vrt.nu/resources/lib/addon.py", line 271, in play_url
    VRTPlayer().play(dict(video_url=video_url))
  File "/home/travis/build/dagwieers/plugin.video.vrt.nu/resources/lib/vrtplayer.py", line 361, in play
    stream = _streamservice.get_stream(video)
  File "/home/travis/build/dagwieers/plugin.video.vrt.nu/resources/lib/streamservice.py", line 198, in get_stream
    stream_json = self._get_stream_json(api_data, roaming)
  File "/home/travis/build/dagwieers/plugin.video.vrt.nu/resources/lib/streamservice.py", line 152, in _get_stream_json
    playertoken = self._tokenresolver.get_playertoken(token_url, token_variant='live', roaming=roaming)
  File "/home/travis/build/dagwieers/plugin.video.vrt.nu/resources/lib/tokenresolver.py", line 66, in get_playertoken
    token = self._get_new_playertoken(token_url, headers, token_variant)
  File "/home/travis/build/dagwieers/plugin.video.vrt.nu/resources/lib/tokenresolver.py", line 131, in _get_new_playertoken
    playertoken = loads(to_unicode(urlopen(req).read()))
  File "/opt/python/3.6.7/lib/python3.6/urllib/request.py", line 223, in urlopen
    return opener.open(url, data, timeout)
  File "/opt/python/3.6.7/lib/python3.6/urllib/request.py", line 532, in open
    response = meth(req, response)
  File "/opt/python/3.6.7/lib/python3.6/urllib/request.py", line 642, in http_response
    'http', request, response, code, msg, hdrs)
  File "/opt/python/3.6.7/lib/python3.6/urllib/request.py", line 570, in error
    return self._call_chain(*args)
  File "/opt/python/3.6.7/lib/python3.6/urllib/request.py", line 504, in _call_chain
    result = func(*args)
  File "/opt/python/3.6.7/lib/python3.6/urllib/request.py", line 650, in http_error_default
    raise HTTPError(req.full_url, code, msg, hdrs, fp)
urllib.error.HTTPError: HTTP Error 504: Gateway Time-out
----------------------------------------------------------------------
Ran 123 tests in 343.550s
FAILED (errors=1)

@dagwieers
Copy link
Collaborator Author

dagwieers commented Nov 18, 2019

urllib.error.HTTPError: HTTP Error 500: Internal Server Error

This error occurred during integration tests on Travis CI:
https://travis-ci.org/pietje666/plugin.video.vrt.nu/jobs/577982967
https://travis-ci.org/pietje666/plugin.video.vrt.nu/jobs/577983676

Still during the same VRT NU infrastructure issues, we see this as well:

======================================================================
ERROR: test_follow_unfollow (test.test_favorites.TestFavorites)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/travis/build/pietje666/plugin.video.vrt.nu/test/test_favorites.py", line 93, in test_follow_unfollow
    self._favorites.follow(program=program, title=program_title)
  File "/home/travis/build/pietje666/plugin.video.vrt.nu/resources/lib/favorites.py", line 108, in follow
    succeeded = self.update(program, title, True)
  File "/home/travis/build/pietje666/plugin.video.vrt.nu/resources/lib/favorites.py", line 87, in update
    result = urlopen(req)
  File "/opt/python/3.7.1/lib/python3.7/urllib/request.py", line 222, in urlopen
    return opener.open(url, data, timeout)
  File "/opt/python/3.7.1/lib/python3.7/urllib/request.py", line 531, in open
    response = meth(req, response)
  File "/opt/python/3.7.1/lib/python3.7/urllib/request.py", line 641, in http_response
    'http', request, response, code, msg, hdrs)
  File "/opt/python/3.7.1/lib/python3.7/urllib/request.py", line 569, in error
    return self._call_chain(*args)
  File "/opt/python/3.7.1/lib/python3.7/urllib/request.py", line 503, in _call_chain
    result = func(*args)
  File "/opt/python/3.7.1/lib/python3.7/urllib/request.py", line 649, in http_error_default
    raise HTTPError(req.full_url, code, msg, hdrs, fp)
urllib.error.HTTPError: HTTP Error 500: Internal Server Error

It occurred a second time in another setting:

======================================================================
ERROR: test_play_livestream_by_id_route (__main__.TestRouter)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "test/test_routing.py", line 176, in test_play_livestream_by_id_route
    plugin.run(['plugin://plugin.video.vrt.nu/play/id/vualto_canvas_geo', '0', ''])
  File "/home/travis/build/pietje666/plugin.video.vrt.nu/test/routing.py", line 120, in run
    self._dispatch(path)
  File "/home/travis/build/pietje666/plugin.video.vrt.nu/test/routing.py", line 131, in _dispatch
    view_func(**kwargs)
  File "/home/travis/build/pietje666/plugin.video.vrt.nu/resources/lib/addon.py", line 225, in play_id
    VRTPlayer(kodi).play(dict(video_id=video_id, publication_id=publication_id))
  File "/home/travis/build/pietje666/plugin.video.vrt.nu/resources/lib/vrtplayer.py", line 285, in play
    stream = _streamservice.get_stream(video)
  File "/home/travis/build/pietje666/plugin.video.vrt.nu/resources/lib/streamservice.py", line 195, in get_stream
    stream_json = self._get_stream_json(api_data, roaming)
  File "/home/travis/build/pietje666/plugin.video.vrt.nu/resources/lib/streamservice.py", line 149, in _get_stream_json
    playertoken = self._tokenresolver.get_playertoken(token_url, token_variant='live', roaming=roaming)
  File "/home/travis/build/pietje666/plugin.video.vrt.nu/resources/lib/tokenresolver.py", line 65, in get_playertoken
    token = self._get_new_playertoken(token_url, headers, token_variant)
  File "/home/travis/build/pietje666/plugin.video.vrt.nu/resources/lib/tokenresolver.py", line 126, in _get_new_playertoken
    playertoken = json.load(urlopen(req))
  File "/opt/python/2.7.15/lib/python2.7/urllib2.py", line 154, in urlopen
    return opener.open(url, data, timeout)
  File "/opt/python/2.7.15/lib/python2.7/urllib2.py", line 435, in open
    response = meth(req, response)
  File "/opt/python/2.7.15/lib/python2.7/urllib2.py", line 548, in http_response
    'http', request, response, code, msg, hdrs)
  File "/opt/python/2.7.15/lib/python2.7/urllib2.py", line 473, in error
    return self._call_chain(*args)
  File "/opt/python/2.7.15/lib/python2.7/urllib2.py", line 407, in _call_chain
    result = func(*args)
  File "/opt/python/2.7.15/lib/python2.7/urllib2.py", line 556, in http_error_default
    raise HTTPError(req.get_full_url(), code, msg, hdrs, fp)
HTTPError: HTTP Error 500: Internal Server Error

@dagwieers
Copy link
Collaborator Author

SSLError: ('The read operation timed out',)

I just got this one:

2019-11-19 01:39:22.657 T:1143976832  NOTICE: [plugin.video.vrt.nu] URL post: https://video-user-data.vrt.be/resume_points
2019-11-19 01:39:32.851 T:1143976832   ERROR: EXCEPTION Thrown (PythonToCppException) : -->Python callback/script returned the following error<--
                                             - NOTE: IGNORING THIS CAN LEAD TO MEMORY LEAKS!
                                            Error Type: <class 'ssl.SSLError'>
                                            Error Contents: ('The read operation timed out',)
                                            Traceback (most recent call last):
                                              File "/storage/.kodi/addons/plugin.video.vrt.nu/resources/lib/addon_entry.py", line 8, in <module>
                                                run(argv)
                                              File "/storage/.kodi/addons/plugin.video.vrt.nu/resources/lib/addon.py", line 299, in run
                                                plugin.run(argv)
                                              File "/storage/.kodi/addons/script.module.routing/lib/routing.py", line 130, in run
                                                self._dispatch(self.path)
                                              File "/storage/.kodi/addons/script.module.routing/lib/routing.py", line 141, in _dispatch
                                                view_func(**kwargs)
                                              File "/storage/.kodi/addons/plugin.video.vrt.nu/resources/lib/addon.py", line 96, in favorites_recent
                                                VRTPlayer().show_recent_menu(page=page, use_favorites=True)
                                              File "/storage/.kodi/addons/plugin.video.vrt.nu/resources/lib/vrtplayer.py", line 254, in show_recent_menu
                                                self._resumepoints.refresh(ttl=5 * 60 if use_favorites else 60 * 60)
                                              File "/storage/.kodi/addons/plugin.video.vrt.nu/resources/lib/resumepoints.py", line 51, in refresh
                                                resumepoints_json = load(urlopen(req))
                                              File "/usr/lib/python2.7/urllib2.py", line 154, in urlopen
                                              File "/usr/lib/python2.7/urllib2.py", line 429, in open
                                              File "/usr/lib/python2.7/urllib2.py", line 447, in _open
                                              File "/usr/lib/python2.7/urllib2.py", line 407, in _call_chain
                                              File "/usr/lib/python2.7/urllib2.py", line 1241, in https_open
                                              File "/usr/lib/python2.7/urllib2.py", line 1201, in do_open
                                              File "/usr/lib/python2.7/httplib.py", line 1121, in getresponse
                                              File "/usr/lib/python2.7/httplib.py", line 438, in begin
                                              File "/usr/lib/python2.7/httplib.py", line 394, in _read_status
                                              File "/usr/lib/python2.7/socket.py", line 480, in readline
                                              File "/usr/lib/python2.7/ssl.py", line 754, in recv
                                              File "/usr/lib/python2.7/ssl.py", line 641, in read
                                            SSLError: ('The read operation timed out',)
                                            -->End of Python script error report<--

@dagwieers
Copy link
Collaborator Author

SSLError: ('The read operation timed out',)

This just happened to me again:

2019-11-26 04:07:13.775 T:1268081536  NOTICE: [plugin.video.vrt.nu] Access: plugin://plugin.video.vrt.nu/play/id/vid-000ffb83-0def-415d-bfd1-a6208d836420/pbs-pub-29fc2903-baba-46b0-a481-6d6848ae5bc7
2019-11-26 04:07:13.851 T:1268081536  NOTICE: [plugin.video.vrt.nu] Got cached token '/storage/.kodi/userdata/addon_data/plugin.video.vrt.nu/tokens/ondemand_vrtPlayerToken.tkn'
2019-11-26 04:07:13.869 T:1268081536  NOTICE: [plugin.video.vrt.nu] URL get: https://media-services-public.vrt.be/vualto-video-aggregator-web/rest/external/v1/videos/pbs-pub-29fc2903-baba-46b0-a481-6d6848ae5bc7$vid-000ffb83-0def-415d-bfd1-a6208d836420?vrtPlayerToken=b10@3897fdc9a30ec05ef002163377571c9277abca3d75129c35abc7962e884932ce&client=vrtvideo@PROD
2019-11-26 04:07:24.122 T:1268081536   ERROR: EXCEPTION Thrown (PythonToCppException) : -->Python callback/script returned the following error<--
                                             - NOTE: IGNORING THIS CAN LEAD TO MEMORY LEAKS!
                                            Error Type: <class 'ssl.SSLError'>
                                            Error Contents: ('The read operation timed out',)
                                            Traceback (most recent call last):
                                              File "/storage/.kodi/addons/plugin.video.vrt.nu/resources/lib/addon_entry.py", line 8, in <module>
                                                run(argv)
                                              File "/storage/.kodi/addons/plugin.video.vrt.nu/resources/lib/addon.py", line 299, in run
                                                plugin.run(argv)
                                              File "/storage/.kodi/addons/script.module.routing/lib/routing.py", line 130, in run
                                                self._dispatch(self.path)
                                              File "/storage/.kodi/addons/script.module.routing/lib/routing.py", line 141, in _dispatch
                                                view_func(**kwargs)
                                              File "/storage/.kodi/addons/plugin.video.vrt.nu/resources/lib/addon.py", line 264, in play_id
                                                VRTPlayer().play(dict(video_id=video_id, publication_id=publication_id))
                                              File "/storage/.kodi/addons/plugin.video.vrt.nu/resources/lib/vrtplayer.py", line 361, in play
                                                stream = _streamservice.get_stream(video)
                                              File "/storage/.kodi/addons/plugin.video.vrt.nu/resources/lib/streamservice.py", line 198, in get_stream
                                                stream_json = self._get_stream_json(api_data, roaming)
                                              File "/storage/.kodi/addons/plugin.video.vrt.nu/resources/lib/streamservice.py", line 164, in _get_stream_json
                                                stream_json = loads(to_unicode(urlopen(api_url).read()))
                                              File "/usr/lib/python2.7/urllib2.py", line 154, in urlopen
                                              File "/usr/lib/python2.7/urllib2.py", line 429, in open
                                              File "/usr/lib/python2.7/urllib2.py", line 447, in _open
                                              File "/usr/lib/python2.7/urllib2.py", line 407, in _call_chain
                                              File "/usr/lib/python2.7/urllib2.py", line 1241, in https_open
                                              File "/usr/lib/python2.7/urllib2.py", line 1201, in do_open
                                              File "/usr/lib/python2.7/httplib.py", line 1121, in getresponse
                                              File "/usr/lib/python2.7/httplib.py", line 438, in begin
                                              File "/usr/lib/python2.7/httplib.py", line 394, in _read_status
                                              File "/usr/lib/python2.7/socket.py", line 480, in readline
                                              File "/usr/lib/python2.7/ssl.py", line 754, in recv
                                              File "/usr/lib/python2.7/ssl.py", line 641, in read
                                            SSLError: ('The read operation timed out',)
                                            -->End of Python script error report<--
2019-11-26 04:07:24.166 T:1937587632   ERROR: Playlist Player: skipping unplayable item: 0, path [plugin://plugin.video.vrt.nu/play/id/vid-000ffb83-0def-415d-bfd1-a6208d836420/pbs-pub-29fc2903-baba-46b0-a481-6d6848ae5bc7]

@dagwieers
Copy link
Collaborator Author

Exception: Playing stream returned: HTTP Error 415: Unsupported Media Type

This happened during integration tests on Travis CI.

======================================================================
ERROR: test_play_latestepisode_route (test.test_routing.TestRouting)
Play last episode method: /play/lastepisode/<program>
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/travis/build/pietje666/plugin.video.vrt.nu/test/test_routing.py", line 230, in test_play_latestepisode_route
    addon.run(['plugin://plugin.video.vrt.nu/play/latest/het-journaal', '0', ''])
  File "/home/travis/build/pietje666/plugin.video.vrt.nu/resources/lib/addon.py", line 298, in run
    plugin.run(argv)
  File "/home/travis/virtualenv/python2.7.15/lib/python2.7/site-packages/routing.py", line 130, in run
    self._dispatch(self.path)
  File "/home/travis/virtualenv/python2.7.15/lib/python2.7/site-packages/routing.py", line 141, in _dispatch
    view_func(**kwargs)
  File "/home/travis/build/pietje666/plugin.video.vrt.nu/resources/lib/addon.py", line 277, in play_latest
    VRTPlayer().play_latest_episode(program=program)
  File "/home/travis/build/pietje666/plugin.video.vrt.nu/resources/lib/vrtplayer.py", line 327, in play_latest_episode
    self.play(video)
  File "/home/travis/build/pietje666/plugin.video.vrt.nu/resources/lib/vrtplayer.py", line 363, in play
    play(stream, video.get('listitem'))
  File "/home/travis/build/pietje666/plugin.video.vrt.nu/resources/lib/kodiutils.py", line 279, in play
    xbmcplugin.setResolvedUrl(plugin.handle, bool(stream.stream_url), listitem=play_item)
  File "/home/travis/build/pietje666/plugin.video.vrt.nu/test/xbmcplugin.py", line 112, in setResolvedUrl
    log('Playing stream returned: %s' % exc, LOGFATAL)
  File "/home/travis/build/pietje666/plugin.video.vrt.nu/test/xbmc.py", line 228, in log
    raise Exception(msg)
Exception: Playing stream returned: HTTP Error 415: Unsupported Media Type

@dagwieers
Copy link
Collaborator Author

dagwieers commented Dec 1, 2019

urllib.error.HTTPError: HTTP Error 401: Unauthorized

This happened during integration tests on Travis CI.
https://travis-ci.org/pietje666/plugin.video.vrt.nu/jobs/618383176
https://travis-ci.org/pietje666/plugin.video.vrt.nu/jobs/618502852
https://travis-ci.org/dagwieers/plugin.video.vrt.nu/jobs/619100944

======================================================================
ERROR: test_httpcaching_disabled (test.test_settings.TestSettings)
Test without http caching
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/travis/build/dagwieers/plugin.video.vrt.nu/test/test_settings.py", line 45, in test_httpcaching_disabled
    plugin.run(['plugin://plugin.video.vrt.nu/offline', '0', ''])
  File "/home/travis/virtualenv/python3.5.6/lib/python3.5/site-packages/routing.py", line 130, in run
    self._dispatch(self.path)
  File "/home/travis/virtualenv/python3.5.6/lib/python3.5/site-packages/routing.py", line 141, in _dispatch
    view_func(**kwargs)
  File "/home/travis/build/dagwieers/plugin.video.vrt.nu/resources/lib/addon.py", line 191, in offline
    VRTPlayer().show_offline_menu(page=page)
  File "/home/travis/build/dagwieers/plugin.video.vrt.nu/resources/lib/vrtplayer.py", line 277, in show_offline_menu
    self._favorites.refresh(ttl=5 * 60 if use_favorites else 60 * 60)
  File "/home/travis/build/dagwieers/plugin.video.vrt.nu/resources/lib/favorites.py", line 50, in refresh
    favorites_json = loads(to_unicode(urlopen(req).read()))
  File "/opt/python/3.5.6/lib/python3.5/urllib/request.py", line 163, in urlopen
    return opener.open(url, data, timeout)
  File "/opt/python/3.5.6/lib/python3.5/urllib/request.py", line 472, in open
    response = meth(req, response)
  File "/opt/python/3.5.6/lib/python3.5/urllib/request.py", line 582, in http_response
    'http', request, response, code, msg, hdrs)
  File "/opt/python/3.5.6/lib/python3.5/urllib/request.py", line 510, in error
    return self._call_chain(*args)
  File "/opt/python/3.5.6/lib/python3.5/urllib/request.py", line 444, in _call_chain
    result = func(*args)
  File "/opt/python/3.5.6/lib/python3.5/urllib/request.py", line 590, in http_error_default
    raise HTTPError(req.full_url, code, msg, hdrs, fp)
urllib.error.HTTPError: HTTP Error 401: Unauthorized

@dagwieers
Copy link
Collaborator Author

dagwieers commented Dec 1, 2019

urllib.error.URLError: <urlopen error Remote end closed connection without response>

This happened during integration tests on Travis CI:
https://travis-ci.org/pietje666/plugin.video.vrt.nu/jobs/606606474
https://travis-ci.org/dagwieers/plugin.video.vrt.nu/jobs/607144469
https://travis-ci.org/dagwieers/plugin.video.vrt.nu/jobs/617979167
https://travis-ci.org/dagwieers/plugin.video.vrt.nu/jobs/618807514
https://travis-ci.org/dagwieers/plugin.video.vrt.nu/jobs/619358840
https://travis-ci.org/pietje666/plugin.video.vrt.nu/jobs/631726974

======================================================================
ERROR: test_az_menu (test.test_proxy.TestProxy)
Test Programs menu
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/opt/python/3.7.1/lib/python3.7/urllib/request.py", line 1317, in do_open
    encode_chunked=req.has_header('Transfer-encoding'))
  File "/opt/python/3.7.1/lib/python3.7/http/client.py", line 1229, in request
    self._send_request(method, url, body, headers, encode_chunked)
  File "/opt/python/3.7.1/lib/python3.7/http/client.py", line 1275, in _send_request
    self.endheaders(body, encode_chunked=encode_chunked)
  File "/opt/python/3.7.1/lib/python3.7/http/client.py", line 1224, in endheaders
    self._send_output(message_body, encode_chunked=encode_chunked)
  File "/opt/python/3.7.1/lib/python3.7/http/client.py", line 1016, in _send_output
    self.send(msg)
  File "/opt/python/3.7.1/lib/python3.7/http/client.py", line 956, in send
    self.connect()
  File "/opt/python/3.7.1/lib/python3.7/http/client.py", line 1384, in connect
    super().connect()
  File "/opt/python/3.7.1/lib/python3.7/http/client.py", line 932, in connect
    self._tunnel()
  File "/opt/python/3.7.1/lib/python3.7/http/client.py", line 906, in _tunnel
    (version, code, message) = response._read_status()
  File "/opt/python/3.7.1/lib/python3.7/http/client.py", line 265, in _read_status
    raise RemoteDisconnected("Remote end closed connection without"
http.client.RemoteDisconnected: Remote end closed connection without response
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
  File "/home/travis/build/dagwieers/plugin.video.vrt.nu/test/test_proxy.py", line 51, in test_az_menu
    plugin.run(['plugin://plugin.video.vrt.nu/programs', '0', ''])
  File "/home/travis/virtualenv/python3.7.1/lib/python3.7/site-packages/routing.py", line 130, in run
    self._dispatch(self.path)
  File "/home/travis/virtualenv/python3.7.1/lib/python3.7/site-packages/routing.py", line 141, in _dispatch
    view_func(**kwargs)
  File "/home/travis/build/dagwieers/plugin.video.vrt.nu/resources/lib/addon.py", line 152, in programs
    VRTPlayer().show_tvshow_menu()
  File "/home/travis/build/dagwieers/plugin.video.vrt.nu/resources/lib/vrtplayer.py", line 187, in show_tvshow_menu
    self._resumepoints.refresh(ttl=ttl('direct' if use_favorites else 'indirect'))
  File "/home/travis/build/dagwieers/plugin.video.vrt.nu/resources/lib/resumepoints.py", line 51, in refresh
    resumepoints_json = loads(to_unicode(urlopen(req).read()))
  File "/opt/python/3.7.1/lib/python3.7/urllib/request.py", line 222, in urlopen
    return opener.open(url, data, timeout)
  File "/opt/python/3.7.1/lib/python3.7/urllib/request.py", line 525, in open
    response = self._open(req, data)
  File "/opt/python/3.7.1/lib/python3.7/urllib/request.py", line 543, in _open
    '_open', req)
  File "/opt/python/3.7.1/lib/python3.7/urllib/request.py", line 503, in _call_chain
    result = func(*args)
  File "/opt/python/3.7.1/lib/python3.7/urllib/request.py", line 1360, in https_open
    context=self._context, check_hostname=self._check_hostname)
  File "/opt/python/3.7.1/lib/python3.7/urllib/request.py", line 1319, in do_open
    raise URLError(err)
urllib.error.URLError: <urlopen error Remote end closed connection without response>
----------------------------------------------------------------------
Ran 123 tests in 120.213s
FAILED (errors=1)

@dagwieers
Copy link
Collaborator Author

dagwieers commented Dec 1, 2019

urllib.error.HTTPError: HTTP Error 500:

The following error occurred during integration tests on Travis CI:
https://travis-ci.org/dagwieers/plugin.video.vrt.nu/jobs/617953851

======================================================================
ERROR: test_manage_favorites_route (test.test_routing.TestRouting)
Manage favorites method: /favorites/manage
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/travis/build/dagwieers/plugin.video.vrt.nu/test/test_routing.py", line 184, in test_manage_favorites_route
    addon.run(['plugin://plugin.video.vrt.nu/favorites/manage', '0', ''])
  File "/home/travis/build/dagwieers/plugin.video.vrt.nu/resources/lib/addon.py", line 299, in run
    plugin.run(argv)
  File "/home/travis/virtualenv/python3.7.1/lib/python3.7/site-packages/routing.py", line 130, in run
    self._dispatch(self.path)
  File "/home/travis/virtualenv/python3.7.1/lib/python3.7/site-packages/routing.py", line 141, in _dispatch
    view_func(**kwargs)
  File "/home/travis/build/dagwieers/plugin.video.vrt.nu/resources/lib/addon.py", line 119, in favorites_manage
    Favorites().manage()
  File "/home/travis/build/dagwieers/plugin.video.vrt.nu/resources/lib/favorites.py", line 140, in manage
    self.refresh(ttl=0)
  File "/home/travis/build/dagwieers/plugin.video.vrt.nu/resources/lib/favorites.py", line 39, in refresh
    xvrttoken = TokenResolver().get_xvrttoken(token_variant='user')
  File "/home/travis/build/dagwieers/plugin.video.vrt.nu/resources/lib/tokenresolver.py", line 79, in get_xvrttoken
    token = self._get_new_user_xvrttoken()
  File "/home/travis/build/dagwieers/plugin.video.vrt.nu/resources/lib/tokenresolver.py", line 241, in _get_new_user_xvrttoken
    opener.open(self._VRT_LOGIN_URL, data=data)
  File "/opt/python/3.7.1/lib/python3.7/urllib/request.py", line 531, in open
    response = meth(req, response)
  File "/opt/python/3.7.1/lib/python3.7/urllib/request.py", line 641, in http_response
    'http', request, response, code, msg, hdrs)
  File "/opt/python/3.7.1/lib/python3.7/urllib/request.py", line 569, in error
    return self._call_chain(*args)
  File "/opt/python/3.7.1/lib/python3.7/urllib/request.py", line 503, in _call_chain
    result = func(*args)
  File "/opt/python/3.7.1/lib/python3.7/urllib/request.py", line 649, in http_error_default
    raise HTTPError(req.full_url, code, msg, hdrs, fp)
urllib.error.HTTPError: HTTP Error 500: 
----------------------------------------------------------------------
Ran 123 tests in 250.884s
FAILED (errors=1)

@dagwieers
Copy link
Collaborator Author

URLError: <urlopen error [Errno -2] Name or service not known>

I got the below error during playback when my Telenet internet connection suddenly disappeared. Cable modem was gone for a few minutes, when it returned, my connection took some more minutes to come back. So I assume there was some Telenet maintenance going on.

The events that happened:

  • Playback stopped, video froze
  • Only after a minute it responded to me pressing Stop or Back button
  • Then I got the typical Kodi error notification
  • And I ended upon the main video menu
2019-12-05 01:17:04.621 T:1239409536   ERROR: CCurlFile::FillBuffer - Failed: Timeout was reached(28)
2019-12-05 01:17:04.622 T:1239409536   ERROR: CCurlFile::Open failed with code 0 for https://ondemand-cf-vrt.akamaized.net/content/vod/vid-0c67065e-3aa5-4c27-ab5a-38cb14d9e72e-CDN_4/vid-0c67065e-3aa5-4c27-ab5a-38cb14d9e72e-CDN_4_nodrm_ce19bbfc-c2b7-4d41-92c0-848068b8543f.ism/dash/vid-0c67065e-3aa5-4c27-ab5a-38cb14d9e72e-CDN_4_nodrm_ce19bbfc-c2b7-4d41-92c0-848068b8543f-video=1996955-1119600.dash:
2019-12-05 01:17:24.645 T:1222624128   ERROR: CCurlFile::FillBuffer - Failed: Timeout was reached(28)
2019-12-05 01:17:24.645 T:1222624128   ERROR: CCurlFile::Open failed with code 0 for https://ondemand-cf-vrt.akamaized.net/content/vod/vid-0c67065e-3aa5-4c27-ab5a-38cb14d9e72e-CDN_4/vid-0c67065e-3aa5-4c27-ab5a-38cb14d9e72e-CDN_4_nodrm_ce19bbfc-c2b7-4d41-92c0-848068b8543f.ism/dash/vid-0c67065e-3aa5-4c27-ab5a-38cb14d9e72e-CDN_4_nodrm_ce19bbfc-c2b7-4d41-92c0-848068b8543f-audio=96000-89568256.dash:
2019-12-05 01:17:27.251 T:1937215504  NOTICE: CVideoPlayer::CloseFile()
2019-12-05 01:17:27.252 T:1937215504  NOTICE: VideoPlayer: waiting for threads to exit
2019-12-05 01:17:27.369 T:1197446016  NOTICE: [plugin.video.vrt.nu] [PlayerInfo 7412] Event onThreadExit
2019-12-05 01:17:27.381 T:1197446016  NOTICE: [plugin.video.vrt.nu] Cached token '/storage/.kodi/userdata/addon_data/plugin.video.vrt.nu/tokens/user_XVRTToken.tkn' deleted
2019-12-05 01:17:27.382 T:1197446016  NOTICE: [plugin.video.vrt.nu] Delete file '/storage/.kodi/userdata/addon_data/plugin.video.vrt.nu/tokens/user_XVRTToken.tkn'.
2019-12-05 01:17:27.391 T:1197446016  NOTICE: [plugin.video.vrt.nu] Got cached token '/storage/.kodi/userdata/addon_data/plugin.video.vrt.nu/tokens/vrtloginrt.tkn'
2019-12-05 01:17:27.396 T:1197446016  NOTICE: [plugin.video.vrt.nu] URL get: https://token.vrt.be/refreshtoken
2019-12-05 01:17:27.849 T:1539265408   ERROR: EXCEPTION: XBMC is not playing any file
2019-12-05 01:17:27.853 T:1539265408  NOTICE: [service.upnext] UpNextMonitor -> No file is playing - stop up next tracking
2019-12-05 01:17:36.080 T:1197446016   ERROR: Exception in thread StreamPosition:
                                            Traceback (most recent call last):
                                              File "/usr/lib/python2.7/threading.py", line 801, in __bootstrap_inner
                                              File "/usr/lib/python2.7/threading.py", line 754, in run
                                              File "/storage/.kodi/addons/plugin.video.vrt.nu/resources/lib/playerinfo.py", line 152, in stream_position
                                                self.onThreadExit()
                                              File "/storage/.kodi/addons/plugin.video.vrt.nu/resources/lib/playerinfo.py", line 144, in onThreadExit
                                                self.push_position(position=self.last_pos, total=self.total)
                                              File "/storage/.kodi/addons/plugin.video.vrt.nu/resources/lib/playerinfo.py", line 212, in push_position
                                                asynchronous=True
                                              File "/storage/.kodi/addons/plugin.video.vrt.nu/resources/lib/resumepoints.py", line 70, in update
                                                self.refresh(ttl=0)
                                              File "/storage/.kodi/addons/plugin.video.vrt.nu/resources/lib/resumepoints.py", line 40, in refresh
                                                xvrttoken = TokenResolver().get_xvrttoken(token_variant='user')
                                              File "/storage/.kodi/addons/plugin.video.vrt.nu/resources/lib/tokenresolver.py", line 77, in get_xvrttoken
                                                token = self._get_fresh_token(refresh_token, 'X-VRT-Token', token_variant=token_variant)
                                              File "/storage/.kodi/addons/plugin.video.vrt.nu/resources/lib/tokenresolver.py", line 261, in _get_fresh_token
                                                opener.open(req)
                                              File "/usr/lib/python2.7/urllib2.py", line 429, in open
                                              File "/usr/lib/python2.7/urllib2.py", line 447, in _open
                                              File "/usr/lib/python2.7/urllib2.py", line 407, in _call_chain
                                              File "/usr/lib/python2.7/urllib2.py", line 1241, in https_open
                                              File "/usr/lib/python2.7/urllib2.py", line 1198, in do_open
                                            URLError: <urlopen error [Errno -2] Name or service not known>
2019-12-05 01:17:44.673 T:1205838720   ERROR: CCurlFile::FillBuffer - Failed: Timeout was reached(28)
2019-12-05 01:17:44.673 T:1205838720   ERROR: CCurlFile::Open failed with code 0 for https://ondemand-cf-vrt.akamaized.net/content/vod/vid-0c67065e-3aa5-4c27-ab5a-38cb14d9e72e-CDN_4/vid-0c67065e-3aa5-4c27-ab5a-38cb14d9e72e-CDN_4_nodrm_ce19bbfc-c2b7-4d41-92c0-848068b8543f.ism/dash/vid-0c67065e-3aa5-4c27-ab5a-38cb14d9e72e-CDN_4_nodrm_ce19bbfc-c2b7-4d41-92c0-848068b8543f-textstream_dut=1000-1920000.dash:
2019-12-05 01:17:44.674 T:1247802240  NOTICE: CVideoPlayer::OnExit()
2019-12-05 01:17:44.674 T:1247802240  NOTICE: Closing stream player 1
2019-12-05 01:17:44.676 T:1214231424  NOTICE: thread end: OMXPlayerAudio::OnExit()
2019-12-05 01:17:44.752 T:1247802240  NOTICE: Closing stream player 2
2019-12-05 01:17:44.754 T:1231016832   ERROR: OMXPlayerVideo: Got MSGQ_IS_ERROR(-1) Aborting
2019-12-05 01:17:44.755 T:1231016832  NOTICE: thread end: video_thread
2019-12-05 01:17:44.796 T:1247802240  NOTICE: Closing stream player 3
2019-12-05 01:17:44.803 T:1937215504  NOTICE: VideoPlayer: finished waiting
2019-12-05 01:17:44.803 T:1937215504  NOTICE: CVideoPlayer::CloseFile()
2019-12-05 01:17:44.803 T:1937215504  NOTICE: VideoPlayer: waiting for threads to exit
2019-12-05 01:17:44.803 T:1937215504  NOTICE: VideoPlayer: finished waiting
2019-12-05 01:17:45.141 T:1937215504   ERROR: Control 55 in window 10025 has been asked to focus, but it can't
2019-12-05 01:17:45.862 T:1564443520   ERROR: Previous line repeats 2 times.
2019-12-05 01:17:45.862 T:1564443520  NOTICE: [plugin.video.vrt.nu] [PlayerInfo 7412] Event onPlayBackStopped
2019-12-05 01:17:46.677 T:1214231424  NOTICE: [plugin.video.vrt.nu] Access: plugin://plugin.video.vrt.nu/favorites/recent
2019-12-05 01:17:46.855 T:1214231424  NOTICE: [plugin.video.vrt.nu] Got cached token '/storage/.kodi/userdata/addon_data/plugin.video.vrt.nu/tokens/vrtloginrt.tkn'
2019-12-05 01:17:46.857 T:1214231424  NOTICE: [plugin.video.vrt.nu] URL get: https://token.vrt.be/refreshtoken
2019-12-05 01:17:46.923 T:1214231424   ERROR: EXCEPTION Thrown (PythonToCppException) : -->Python callback/script returned the following error<--
                                             - NOTE: IGNORING THIS CAN LEAD TO MEMORY LEAKS!
                                            Error Type: <class 'urllib2.URLError'>
                                            Error Contents: <urlopen error [Errno -2] Name or service not known>
                                            Traceback (most recent call last):
                                              File "/storage/.kodi/addons/plugin.video.vrt.nu/resources/lib/addon_entry.py", line 14, in <module>
                                                run(argv)
                                              File "/storage/.kodi/addons/plugin.video.vrt.nu/resources/lib/addon.py", line 298, in run
                                                plugin.run(argv)
                                              File "/storage/.kodi/addons/script.module.routing/lib/routing.py", line 130, in run
                                                self._dispatch(self.path)
                                              File "/storage/.kodi/addons/script.module.routing/lib/routing.py", line 141, in _dispatch
                                                view_func(**kwargs)
                                              File "/storage/.kodi/addons/plugin.video.vrt.nu/resources/lib/addon.py", line 95, in favorites_recent
                                                VRTPlayer().show_recent_menu(page=page, use_favorites=True)
                                              File "/storage/.kodi/addons/plugin.video.vrt.nu/resources/lib/vrtplayer.py", line 252, in show_recent_menu
                                                self._favorites.refresh(ttl=ttl('direct' if use_favorites else 'indirect'))
                                              File "/storage/.kodi/addons/plugin.video.vrt.nu/resources/lib/favorites.py", line 39, in refresh
                                                xvrttoken = TokenResolver().get_xvrttoken(token_variant='user')
                                              File "/storage/.kodi/addons/plugin.video.vrt.nu/resources/lib/tokenresolver.py", line 77, in get_xvrttoken
                                                token = self._get_fresh_token(refresh_token, 'X-VRT-Token', token_variant=token_variant)
                                              File "/storage/.kodi/addons/plugin.video.vrt.nu/resources/lib/tokenresolver.py", line 261, in _get_fresh_token
                                                opener.open(req)
                                              File "/usr/lib/python2.7/urllib2.py", line 429, in open
                                              File "/usr/lib/python2.7/urllib2.py", line 447, in _open
                                              File "/usr/lib/python2.7/urllib2.py", line 407, in _call_chain
                                              File "/usr/lib/python2.7/urllib2.py", line 1241, in https_open
                                              File "/usr/lib/python2.7/urllib2.py", line 1198, in do_open
                                            URLError: <urlopen error [Errno -2] Name or service not known>
                                            -->End of Python script error report<--
2019-12-05 01:17:47.159 T:1231016832   ERROR: GetDirectory - Error getting plugin://plugin.video.vrt.nu/favorites/recent
2019-12-05 01:17:47.163 T:1937215504   ERROR: CGUIMediaWindow::GetDirectory(plugin://plugin.video.vrt.nu/favorites/recent) failed

@dagwieers
Copy link
Collaborator Author

HTTPError: HTTP Error 404: Not Found

During integration tests, this happened.

======================================================================
ERROR: test_get_ondemand_stream_from_url_gets_stream_does_not_crash (test.test_streamservice.TestStreamService)
Test getting stream from URL does not crash
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/dag/home-made/plugin.video.vrt.nu/test/test_streamservice.py", line 60, in test_get_ondemand_stream_from_url_gets_stream_does_not_crash
    stream = self._streamservice.get_stream(video)
  File "/home/dag/home-made/plugin.video.vrt.nu/resources/lib/streamservice.py", line 188, in get_stream
    api_data = self._get_api_data(video)
  File "/home/dag/home-made/plugin.video.vrt.nu/resources/lib/streamservice.py", line 106, in _get_api_data
    api_data = self._webscrape_api_data(video_url) or ApiData(self._CLIENT, self._VUALTO_API_URL, video_id, '', True)
  File "/home/dag/home-made/plugin.video.vrt.nu/resources/lib/streamservice.py", line 113, in _webscrape_api_data
    html_page = urlopen(video_url).read()
  File "/usr/lib64/python2.7/urllib2.py", line 154, in urlopen
    return opener.open(url, data, timeout)
  File "/usr/lib64/python2.7/urllib2.py", line 437, in open
    response = meth(req, response)
  File "/usr/lib64/python2.7/urllib2.py", line 550, in http_response
    'http', request, response, code, msg, hdrs)
  File "/usr/lib64/python2.7/urllib2.py", line 475, in error
    return self._call_chain(*args)
  File "/usr/lib64/python2.7/urllib2.py", line 409, in _call_chain
    result = func(*args)
  File "/usr/lib64/python2.7/urllib2.py", line 558, in http_error_default
    raise HTTPError(req.get_full_url(), code, msg, hdrs, fp)
HTTPError: HTTP Error 404: Not Found

@dagwieers
Copy link
Collaborator Author

dagwieers commented Jan 21, 2020

Whoops something went wrong, check the kodi log for more details.

I got the below error after playing another recent item in the Most recent listing. What I think happened is that this episode of De ideale wereld was ongoing when the listing was made, and it was finished (and re-cut) after playing another item. So the listed item was no longer available because it was re-cut into another video.

My indirect HTTP cache time-to-live was set to 120 minutes (2 hours) for testing purposes and this obviously is not a good value. It would probably be better to lower this to e.g. 30 or 45 minutes so the probability of the menu being rebuilt after playing an item is a lot higher. (Currently it is 60 minutes)

2020-01-21 23:48:44.997 T:1121973120  NOTICE: [plugin.video.vrt.nu] Access: plugin://plugin.video.vrt.nu/play/id/vid-5db990ee-fd54-467c-8a18-a809dd124fe8/pbs-pub-e4395db9-6fee-4574-961c-4430e2f47fef
2020-01-21 23:48:45.016 T:1121973120  NOTICE: [plugin.video.vrt.nu] Cached token '/storage/.kodi/userdata/addon_data/plugin.video.vrt.nu/tokens/ondemand_vrtPlayerToken.tkn' deleted
2020-01-21 23:48:45.017 T:1121973120  NOTICE: [plugin.video.vrt.nu] Delete file '/storage/.kodi/userdata/addon_data/plugin.video.vrt.nu/tokens/ondemand_vrtPlayerToken.tkn'.
2020-01-21 23:48:45.022 T:1121973120  NOTICE: [plugin.video.vrt.nu] Got cached token '/storage/.kodi/userdata/addon_data/plugin.video.vrt.nu/tokens/XVRTToken.tkn'
2020-01-21 23:48:45.024 T:1121973120  NOTICE: [plugin.video.vrt.nu] URL post: https://media-services-public.vrt.be/vualto-video-aggregator-web/rest/external/v1/tokens
2020-01-21 23:48:45.025 T:1121973120  NOTICE: [plugin.video.vrt.nu] URL post data:
2020-01-21 23:48:45.307 T:1121973120  NOTICE: [plugin.video.vrt.nu] URL get: https://media-services-public.vrt.be/vualto-video-aggregator-web/rest/external/v1/videos/pbs-pub-e4395db9-6fee-4574-961c-4430e2f47fef$vid-5db990ee-fd54-467c-8a18-a809dd124fe8?vrtPlayerToken=b10@08438b982c3888a6f2f8c4a17462da64ac86b5d23c1a548a96ab23eae2e3b977&client=vrtvideo@PROD
2020-01-21 23:48:45.610 T:1121973120   ERROR: [plugin.video.vrt.nu] No video found for pbs-pub-e4395db9-6fee-4574-961c-4430e2f47fef$vid-5db990ee-fd54-467c-8a18-a809dd124fe8

After refreshing the menu, the listed item did play correctly (using a different video id).

2020-01-21 23:52:42.195 T:1121973120  NOTICE: [plugin.video.vrt.nu] Access: plugin://plugin.video.vrt.nu/play/id/vid-4af8d7cc-3f4c-4802-bbba-283e789dbf47/pbs-pub-e4395db9-6fee-4574-961c-4430e2f47fef
2020-01-21 23:52:42.217 T:1121973120  NOTICE: [plugin.video.vrt.nu] Got cached token '/storage/.kodi/userdata/addon_data/plugin.video.vrt.nu/tokens/ondemand_vrtPlayerToken.tkn'
2020-01-21 23:52:42.218 T:1121973120  NOTICE: [plugin.video.vrt.nu] URL get: https://media-services-public.vrt.be/vualto-video-aggregator-web/rest/external/v1/videos/pbs-pub-e4395db9-6fee-4574-961c-4430e2f47fef$vid-4af8d7cc-3f4c-4802-bbba-283e789dbf47?vrtPlayerToken=b10@08438b982c3888a6f2f8c4a17462da64ac86b5d23c1a548a96ab23eae2e3b977&client=vrtvideo@PROD
2020-01-21 23:52:42.495 T:1121973120  NOTICE: [plugin.video.vrt.nu] Protocol: mpeg_dash
2020-01-21 23:52:42.495 T:1121973120   ERROR: EXCEPTION: Invalid setting type
2020-01-21 23:52:42.815 T:1121973120   ERROR: Previous line repeats 1 times.
2020-01-21 23:52:42.815 T:1121973120  NOTICE: [plugin.video.vrt.nu] Play: https://remix-cf-vrt.akamaized.net/remix/2c1f1451-f7f4-4d49-ad36-fb4bdaa11c9f/remix.ism/.mpd
2020-01-21 23:52:42.844 T:1937637952  NOTICE: VideoPlayer::OpenFile: plugin://plugin.video.vrt.nu/play/id/vid-4af8d7cc-3f4c-4802-bbba-283e789dbf47/pbs-pub-e4395db9-6fee-4574-961c-4430e2f47fef
2020-01-21 23:52:42.846 T:1134535552  NOTICE: Creating InputStream
2020-01-21 23:52:42.864 T:1564054400  NOTICE: [plugin.video.vrt.nu] [PlayerInfo 7619] Event onPlayBackStarted
2020-01-21 23:52:43.155 T:1564054400  NOTICE: [plugin.video.vrt.nu] URL get: https://vrtnu-api.vrt.be/search?i=video&facets[videoId]=vid-4af8d7cc-3f4c-4802-bbba-283e789dbf47&size=1
2020-01-21 23:52:43.245 T:1134535552  NOTICE: Creating Demuxer
2020-01-21 23:52:43.246 T:1134535552  NOTICE: Opening stream: 1001 source: 256
2020-01-21 23:52:43.351 T:1134535552  NOTICE: Creating video codec with codec id: 27
2020-01-21 23:52:43.363 T:1134535552  NOTICE: Creating video thread
2020-01-21 23:52:43.363 T:1134535552  NOTICE: Opening stream: 1002 source: 256
2020-01-21 23:52:43.364 T:1151320960  NOTICE: running thread: video_thread
2020-01-21 23:52:43.468 T:1134535552  NOTICE: Finding audio codec for: 86018
2020-01-21 23:52:43.469 T:1134535552  NOTICE: CDVDAudioCodecFFmpeg::Open() Successful opened audio decoder aac
2020-01-21 23:52:43.469 T:1134535552  NOTICE: Creating audio thread
2020-01-21 23:52:43.469 T:1055228800  NOTICE: running thread: CVideoPlayerAudio::Process()
2020-01-21 23:52:43.978 T:1055228800  NOTICE: CDVDAudioCodecFFmpeg::Open() Successful opened audio decoder aac
2020-01-21 23:52:43.979 T:1055228800  NOTICE: Creating audio stream (codec id: 86018, channels: 2, sample rate: 48000, no pass-through)
2020-01-21 23:52:44.300 T:1564054400  NOTICE: [plugin.video.vrt.nu] [PlayerInfo 7619] Event onAVStarted
2020-01-21 23:52:44.310 T:1564054400  NOTICE: [plugin.video.vrt.nu] URL get: https://vrtnu-api.vrt.be/search?i=video&q=De+ideale+wereld&highlight=true&size=300
2020-01-21 23:52:44.628 T:1564054400  NOTICE: [plugin.video.vrt.nu] [Up Next] No api data found for De ideale wereld S2020 voorjaarE3

De publication id stays the same, but the video id does get changed when the video is re-cut.

I just experienced this again:

2020-01-28 23:57:58.123 T:1158673280  NOTICE: [plugin.video.vrt.nu] Access: plugin://plugin.video.vrt.nu/play/id/vid-ac4fb1b2-a1ac-4532-ae3f-05c55443fc22/pbs-pub-3bc172f2-528e-497a-9bb4-4426e5d095f9
2020-01-28 23:57:58.153 T:1158673280  NOTICE: [plugin.video.vrt.nu] Got cached token '/storage/.kodi/userdata/addon_data/plugin.video.vrt.nu/tokens/ondemand_vrtPlayerToken.tkn'
2020-01-28 23:57:58.155 T:1158673280  NOTICE: [plugin.video.vrt.nu] URL get: https://media-services-public.vrt.be/vualto-video-aggregator-web/rest/external/v1/videos/pbs-pub-3bc172f2-528e-497a-9bb4-4426e5d095f9$vid-ac4fb1b2-a1ac-4532-ae3f-05c55443fc22?vrtPlayerToken=b10@66c4609bc42329d2c66a7018dee233fb85ba87f1ee7a62ef519851bd1a8a7ad1&client=vrtvideo@PROD
2020-01-28 23:57:58.335 T:1158673280   ERROR: [plugin.video.vrt.nu] No video found for pbs-pub-3bc172f2-528e-497a-9bb4-4426e5d095f9$vid-ac4fb1b2-a1ac-4532-ae3f-05c55443fc22
2020-01-28 23:57:59.851 T:1936917056   ERROR: Playlist Player: skipping unplayable item: 0, path [plugin://plugin.video.vrt.nu/play/id/vid-ac4fb1b2-a1ac-4532-ae3f-05c55443fc22/pbs-pub-3bc172f2-528e-497a-9bb4-4426e5d095f9]

The solution here is to refresh the listing because it has gone stale. This happens when the listing is showing an item that is still being broadcasted, and refreshing the listing would update the stale item.

Currently the add-on works fine if all is well, however various things
can go wrong which makes the add-on bail out with no proper error.

And since the cause of network errors could be anywhere, we do not want
the end-user to have to dig deep into Kodi logs to find probable
suspects.

So we do want to return the exact error message (i.e. Connection
refused, No route to host, etc.) so the user can relate it to other
possible problems. (WIFI issues, Internet down or unreliable, etc.)
@add-ons add-ons deleted a comment from codecov bot Apr 9, 2020
@dagwieers
Copy link
Collaborator Author

An implementation like this would have helped for #748 so users do not need to go to the Kodi log to have an identifiable error to report.

@dagwieers
Copy link
Collaborator Author

dagwieers commented Aug 30, 2020

HTTP Error 503: Service Unavailable: Back-end server is at capacity

I got a new error message last night, which gives some insight where the problem is.

2020-08-29 23:47:14.677 T:1037038464  NOTICE: [plugin.video.vrt.nu] URL get: https://token.vrt.be/vrtnuinitlogin?provider=site&destination=https://www.vrt.be/vrtnu/
2020-08-29 23:47:29.408 T:1037038464   ERROR: [plugin.video.vrt.nu] HTTP Error 503: Service Unavailable: Back-end server is at capacity

@dagwieers
Copy link
Collaborator Author

HTTP Error 504: GATEWAY_TIMEOUT

And this one is related to the previous error.

2020-08-29 23:47:54.246 T:1049621376  NOTICE: [plugin.video.vrt.nu] URL post: https://accounts.vrt.be/accounts.login
2020-08-29 23:47:54.247 T:1049621376  NOTICE: [plugin.video.vrt.nu] URL post data: targetEnv=jssdk&sessionExpiration=-2&password=<redacted>&APIKey=<redacted>&loginID=<redacted>
2020-08-29 23:47:54.666 T:1049621376  NOTICE: [plugin.video.vrt.nu] URL get: https://token.vrt.be/vrtnuinitlogin?provider=site&destination=https://www.vrt.be/vrtnu/
2020-08-29 23:47:55.991 T:1049621376   ERROR: [plugin.video.vrt.nu] HTTP Error 504: GATEWAY_TIMEOUT

@mediaminister
Copy link
Collaborator

Yesterday evening, there was a problem logging in at VRT NU: https://twitter.com/VRT_NU/status/1299829574759583744

Did you get an ok dialog in Kodi GUI? Which message?

@dagwieers
Copy link
Collaborator Author

It started with a blank statement of an issue that could be related to almost anything, followed by specific error messages like the above in dialogs. But then I still got a generic Kodi error referring me to the logs which was caused by the SSLError in #806

So the communication was a lot better than before.

@dagwieers
Copy link
Collaborator Author

http.client.IncompleteRead: IncompleteRead(3004 bytes read)

Got another weird error in GitHub CI today:

ERROR: test_play_latestepisode_route (tests.test_routing.TestRouting)
----------------------------------------------------------------------
Error: [plugin.video.vrt.nu] Web scraping video attributes failed: HTTP Error 404: Not Found
Traceback (most recent call last):
  File "/opt/hostedtoolcache/Python/3.8.5/x64/lib/python3.8/http/client.py", line 555, in _get_chunk_left
    chunk_left = self._read_next_chunk_size()
  File "/opt/hostedtoolcache/Python/3.8.5/x64/lib/python3.8/http/client.py", line 522, in _read_next_chunk_size
    return int(line, 16)
ValueError: invalid literal for int() with base 16: b''
Error: [plugin.video.vrt.nu] Web scraping video attributes failed: 'NoneType' object has no attribute 'find'

During handling of the above exception, another exception occurred:
Error: [plugin.video.vrt.nu] Web scraping video attributes failed: 'NoneType' object has no attribute 'find'
Traceback (most recent call last):
  File "/opt/hostedtoolcache/Python/3.8.5/x64/lib/python3.8/http/client.py", line 572, in _readall_chunked
    chunk_left = self._get_chunk_left()
  File "/opt/hostedtoolcache/Python/3.8.5/x64/lib/python3.8/http/client.py", line 557, in _get_chunk_left
    raise IncompleteRead(b'')
http.client.IncompleteRead: IncompleteRead(0 bytes read)

During handling of the above exception, another exception occurred:
Traceback (most recent call last):
  File "/home/runner/work/plugin.video.vrt.nu/plugin.video.vrt.nu/tests/test_routing.py", line 241, in test_play_latestepisode_route
    addon.run(['plugin://plugin.video.vrt.nu/play/latest/het-journaal', '0', ''])
  File "/home/runner/work/plugin.video.vrt.nu/plugin.video.vrt.nu/resources/lib/addon.py", line 360, in run
    plugin.run(argv)
  File "/opt/hostedtoolcache/Python/3.8.5/x64/lib/python3.8/site-packages/routing.py", line 130, in run
    self._dispatch(self.path)
  File "/opt/hostedtoolcache/Python/3.8.5/x64/lib/python3.8/site-packages/routing.py", line 141, in _dispatch
    view_func(**kwargs)
  File "/home/runner/work/plugin.video.vrt.nu/plugin.video.vrt.nu/resources/lib/addon.py", line 299, in play_latest
    VRTPlayer().play_latest_episode(program=program)
  File "/home/runner/work/plugin.video.vrt.nu/plugin.video.vrt.nu/resources/lib/vrtplayer.py", line 332, in play_latest_episode
    self.play(video)
  File "/home/runner/work/plugin.video.vrt.nu/plugin.video.vrt.nu/resources/lib/vrtplayer.py", line 375, in play
    stream = _streamservice.get_stream(video)
  File "/home/runner/work/plugin.video.vrt.nu/plugin.video.vrt.nu/resources/lib/streamservice.py", line 257, in get_stream
    stream = self._select_hls_substreams(manifest_url, protocol)
  File "/home/runner/work/plugin.video.vrt.nu/plugin.video.vrt.nu/resources/lib/streamservice.py", line 327, in _select_hls_substreams
    hls_playlist = to_unicode(response.read())
  File "/opt/hostedtoolcache/Python/3.8.5/x64/lib/python3.8/http/client.py", line 465, in read
    return self._readall_chunked()
  File "/opt/hostedtoolcache/Python/3.8.5/x64/lib/python3.8/http/client.py", line 579, in _readall_chunked
    raise IncompleteRead(b''.join(value))
http.client.IncompleteRead: IncompleteRead(3004 bytes read)

@dagwieers
Copy link
Collaborator Author

dagwieers commented Oct 5, 2020

ERROR: CCurlFile::FillBuffer - Failed: Timeout was reached(28)

Tonight I experienced a whole new behaviour for the first time.

2020-10-05 23:44:43.760 T:732951424   ERROR: CCurlFile::FillBuffer - Failed: Timeout was reached(28)
2020-10-05 23:44:43.770 T:732951424   ERROR: CCurlFile::Open failed with code 0 for https://remix-cf.lwc.vrtcdn.be/remix/a6c844df-9913-4725-b286-42c57a7709c6/remix.ism/dash/remix-video=1998764-401430.dash:
2020-10-05 23:45:03.797 T:808444800   ERROR: CCurlFile::FillBuffer - Failed: Timeout was reached(28)
2020-10-05 23:45:03.797 T:808444800   ERROR: CCurlFile::Open failed with code 0 for https://remix-cf.lwc.vrtcdn.be/remix/a6c844df-9913-4725-b286-42c57a7709c6/remix.ism/dash/remix-audio=96000-32160768.dash:
2020-10-05 23:45:23.829 T:874505088   ERROR: CCurlFile::FillBuffer - Failed: Timeout was reached(28)
2020-10-05 23:45:23.829 T:874505088   ERROR: CCurlFile::Open failed with code 0 for https://remix-cf.lwc.vrtcdn.be/remix/a6c844df-9913-4725-b286-42c57a7709c6/remix.ism/dash/remix-textstream_dut=1000-723050.dash:
2020-10-05 23:45:23.972 T:963629952  NOTICE: CVideoPlayer::OnExit()
2020-10-05 23:45:23.972 T:963629952  NOTICE: VideoPlayer: eof, waiting for queues to empty
2020-10-05 23:45:23.973 T:963629952  NOTICE: Closing stream player 1
2020-10-05 23:45:23.980 T:963629952  NOTICE: CDVDMessageQueue(audio)::WaitUntilEmpty
2020-10-05 23:45:23.999 T:781185920  NOTICE: thread end: OMXPlayerAudio::OnExit()
2020-10-05 23:45:24.079 T:963629952  NOTICE: Closing stream player 2
2020-10-05 23:45:24.080 T:963629952  NOTICE: CDVDMessageQueue(video)::WaitUntilEmpty
2020-10-05 23:45:24.080 T:816837504   ERROR: OMXPlayerVideo: Got MSGQ_IS_ERROR(-1) Aborting
2020-10-05 23:45:24.080 T:816837504  NOTICE: thread end: video_thread
2020-10-05 23:45:24.127 T:963629952  NOTICE: Closing stream player 3
2020-10-05 23:45:24.275 T:756007808  NOTICE: [plugin.video.vrt.nu] [PlayerInfo 3196] Event onPlayerExit
2020-10-05 23:45:24.301 T:756007808  NOTICE: [plugin.video.vrt.nu] Got item from cache '/storage/.kodi/userdata/addon_data/plugin.video.vrt.nu/tokens/user_XVRTToken.tkn'

There was no indication of buffering issues on screen, playback stalled and after about 20 seconds you return to the menu and a generic VRT NU pop-up appear that there were network issues. It is possible my RPi had network issues, but it is happening multiple times with 5 to 10 minutes of normal playback. Rebooting did not help.

Playback from Chrome on my laptop worked as expected.

@dagwieers
Copy link
Collaborator Author

http.client.IncompleteRead: IncompleteRead(3004 bytes read)

This was encounter on GitHub CI on Windows using Python 3.8.

...........................................................E.................................................................................
======================================================================
ERROR: test_play_latestepisode_route (tests.test_routing.TestRouting)
Play last episode method: /play/lastepisode/<program>
----------------------------------------------------------------------
Traceback (most recent call last):
  File "c:\hostedtoolcache\windows\python\3.8.6\x64\lib\http\client.py", line 555, in _get_chunk_left
    chunk_left = self._read_next_chunk_size()
  File "c:\hostedtoolcache\windows\python\3.8.6\x64\lib\http\client.py", line 522, in _read_next_chunk_size
    return int(line, 16)
ValueError: invalid literal for int() with base 16: b''

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "c:\hostedtoolcache\windows\python\3.8.6\x64\lib\http\client.py", line 572, in _readall_chunked
    chunk_left = self._get_chunk_left()
  File "c:\hostedtoolcache\windows\python\3.8.6\x64\lib\http\client.py", line 557, in _get_chunk_left
    raise IncompleteRead(b'')
http.client.IncompleteRead: IncompleteRead(0 bytes read)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "D:\a\plugin.video.vrt.nu\plugin.video.vrt.nu\tests\test_routing.py", line 241, in test_play_latestepisode_route
    addon.run(['plugin://plugin.video.vrt.nu/play/latest/het-journaal', '0', ''])
  File "D:\a\plugin.video.vrt.nu\plugin.video.vrt.nu\resources\lib\addon.py", line 360, in run
    plugin.run(argv)
  File "c:\hostedtoolcache\windows\python\3.8.6\x64\lib\site-packages\routing.py", line 130, in run
    self._dispatch(self.path)
  File "c:\hostedtoolcache\windows\python\3.8.6\x64\lib\site-packages\routing.py", line 141, in _dispatch
    view_func(**kwargs)
  File "D:\a\plugin.video.vrt.nu\plugin.video.vrt.nu\resources\lib\addon.py", line 299, in play_latest
    VRTPlayer().play_latest_episode(program=program)
  File "D:\a\plugin.video.vrt.nu\plugin.video.vrt.nu\resources\lib\vrtplayer.py", line 332, in play_latest_episode
    self.play(video)
  File "D:\a\plugin.video.vrt.nu\plugin.video.vrt.nu\resources\lib\vrtplayer.py", line 375, in play
    stream = _streamservice.get_stream(video)
  File "D:\a\plugin.video.vrt.nu\plugin.video.vrt.nu\resources\lib\streamservice.py", line 257, in get_stream
    stream = self._select_hls_substreams(manifest_url, protocol)
  File "D:\a\plugin.video.vrt.nu\plugin.video.vrt.nu\resources\lib\streamservice.py", line 327, in _select_hls_substreams
    hls_playlist = to_unicode(response.read())
  File "c:\hostedtoolcache\windows\python\3.8.6\x64\lib\http\client.py", line 465, in read
    return self._readall_chunked()
  File "c:\hostedtoolcache\windows\python\3.8.6\x64\lib\http\client.py", line 579, in _readall_chunked
    raise IncompleteRead(b''.join(value))
http.client.IncompleteRead: IncompleteRead(3004 bytes read)

Error: [plugin.video.vrt.nu] Web scraping video attributes failed: HTTP Error 404: Not Found
Error: [plugin.video.vrt.nu] Web scraping video attributes failed: 'NoneType' object has no attribute 'find'
Error: [plugin.video.vrt.nu] Web scraping video attributes failed: 'NoneType' object has no attribute 'find'

----------------------------------------------------------------------
Ran 141 tests in 99.986s

FAILED (errors=1)
Error: Process completed with exit code 1.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants