Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: adding aws controller for llm application #31

Merged
merged 24 commits into from
Oct 7, 2024

Conversation

kkiani
Copy link
Contributor

@kkiani kkiani commented Oct 4, 2024

This pull request introduces several new examples and enhancements for working with LLMs (Large Language Models) using different approaches and tools. The changes include new example projects, additional functionality for AWS controllers, and improvements to existing components and resources.

New Example Projects:

  • Dspy Application:

    • Added Pulumi.yaml configuration for a minimal Dspy application.
    • Implemented main application logic in __main__.py using AwsDspyController.
    • Defined a default question handler in applications.py.
    • Created AwsDspyController in controller.py to manage Dspy applications.
    • Updated requirements.txt with necessary dependencies.
  • LLM with OpenAI Client:

    • Added Pulumi.yaml configuration for an OpenAI client application.
    • Implemented main application logic in __main__.py using AwsLlmController.
    • Updated requirements.txt with necessary dependencies.
  • LLM with Simple Requests:

    • Added Pulumi.yaml configuration for a simple LLM application using open-source models.
    • Implemented main application logic in __main__.py using AwsLlmController and requests library.
    • Updated requirements.txt with necessary dependencies.

Enhancements to AWS Controllers:

  • Base Controller Enhancements:

    • Added environment and is_runtime_execution properties to base_controller.py.
  • LLM Controller:

    • Introduced LlmController as a base class for LLM-related controllers.
    • Created AwsLlmController to manage AWS-specific LLM operations, including SSM parameter retrieval and OpenAI client integration.

Improvements to AWS Resources:

  • AwsVllmComponent:
    • Added support for storing endpoint URLs in SSM parameters. [1] [2]
    • Enhanced API Gateway resources to include versioning and specific routes for chat and completions. [1] [2] [3] [4] [5]

examples/advance-llm-with-dspy/Pulumi.yaml Outdated Show resolved Hide resolved
examples/advance-llm-with-dspy/controller.py Outdated Show resolved Hide resolved
examples/advance-llm-with-dspy/controller.py Outdated Show resolved Hide resolved
examples/llm-with-simple-requests/__main__.py Outdated Show resolved Hide resolved
src/damavand/base/controllers/base_controller.py Outdated Show resolved Hide resolved
src/damavand/base/controllers/llm.py Outdated Show resolved Hide resolved
src/damavand/cloud/aws/controllers/llm.py Outdated Show resolved Hide resolved
src/damavand/cloud/aws/resources/vllm_component.py Outdated Show resolved Hide resolved
.test.sh Outdated Show resolved Hide resolved
@kkiani kkiani merged commit cb22aa1 into main Oct 7, 2024
2 checks passed
@kkiani kkiani deleted the feature/aws-llm-controller branch October 7, 2024 12:56
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Integrate dspy as the Application Layer with AWS Controller for LLM Integration
3 participants