Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Unresponsive without internet connection #420

Open
georgeto opened this issue Jan 24, 2021 · 1 comment
Open

Unresponsive without internet connection #420

georgeto opened this issue Jan 24, 2021 · 1 comment
Labels

Comments

@georgeto
Copy link
Contributor

When SIPA loses its Internet access, it becomes unresponsive.
Maybe because it tries to fetch other services, such as the Pycroft backend or PBX?

@lukasjuhrich
Copy link
Collaborator

lukasjuhrich commented Jan 24, 2021

I think I have a hunch going through the logs: They are full of:

Sun Jan 24 20:54:35 2021 - *** uWSGI listen queue of socket "0.0.0.0:5000" (fd: 3) full !!! (101/100) ***
Sun Jan 24 20:54:36 2021 - *** uWSGI listen queue of socket "0.0.0.0:5000" (fd: 3) full !!! (101/100) ***
Sun Jan 24 20:54:37 2021 - *** uWSGI listen queue of socket "0.0.0.0:5000" (fd: 3) full !!! (101/100) ***

Now add the following to the mix:

WSGI app 0 (mountpoint='/sipa') ready in 134 seconds on interpreter 0x55ebba9f86b0 pid: 95268 (default app)

This together with Git fetch failed suggests to me that on startup, SIPA waits for the fetch until it times out. In thea meantime, a lot of requests (even if it's just HEAD / from a proxy) build up and flood the listen queue so the worker commits HARAKIRI.

There are multiple options for this:

  • Quickfix: Figure out whether we can decrease the timeout for the git fetch and increase the listen queue capacity
  • Proper fix: Establish some kind of „health check“ either before all of the start startup, or before trying to fetch the git repo. This may e.g. just be a ping to the git server.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

2 participants