Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Remove Gunicorn and use Uvicorn only for gateway #11

Open
wants to merge 2 commits into
base: main
Choose a base branch
from

Conversation

lizard-boy
Copy link

Pull Request Summary

in kubernetes environment we don't really need multiple workers in the same pod, rather it's simpler to just have kubernetes autoscale the number of pods. based on some internal benchmarks gunicorn has some known load balancing issues, also removing this layer results in less error and better latency

Test Plan and Usage Guide

will run simple load testing for get requests with and without gunicorn

Copy link

@greptile-apps greptile-apps bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

PR Summary

This PR removes Gunicorn and replaces it with Uvicorn for the FastAPI server in a Kubernetes environment, aiming to simplify deployment and potentially improve performance.

  • Updated start_fastapi_server.py to use Uvicorn directly with a high concurrency limit (10000)
  • Removed worker.py, eliminating the custom LaunchWorker class for Gunicorn workers
  • Significantly updated Uvicorn from version 0.17.6 to 0.30.0 in requirements.in and requirements.txt
  • Adjusted server configuration parameters in start_fastapi_server.py for Uvicorn-only setup
  • Removed Gunicorn-related dependencies from requirements.txt

5 file(s) reviewed, 1 comment(s)
Edit PR Review Bot Settings

@@ -38,11 +52,10 @@ def entrypoint():
# We can probably use asyncio since this service is going to be more I/O bound.
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

style: This comment about asyncio seems outdated now that we're using Uvicorn. Consider removing or updating it.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants