Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Create Pulumi Component for Azure Resource Layer to Support vLLM #23

Open
8 tasks
kkiani opened this issue Sep 18, 2024 · 0 comments
Open
8 tasks

Create Pulumi Component for Azure Resource Layer to Support vLLM #23

kkiani opened this issue Sep 18, 2024 · 0 comments
Labels
feature/integration Changes and suggestion for currently integrated or under integration application integration/openllm

Comments

@kkiani
Copy link
Contributor

kkiani commented Sep 18, 2024

Description:

Following the discussion on integrating LLM/Foundation Model AI applications, this issue focuses on building a Pulumi component to define the necessary Azure resources for deploying a vLLM model using Damavand. The goal is to simplify infrastructure setup for developers building AI applications on Azure, using resources like Azure Synapse or a VM backend to host vLLM models.

Key Tasks:

  • Implement the Pulumi component to provision Azure resources required for vLLM, such as VMs, storage accounts, and networking.
  • Ensure the component follows best security practices.

Resources:

  • Research Azure AI and other potential solutions for hosting vLLM models on Azure.
  • Explore open-source tools or tutorials that support AI model hosting on Azure infrastructure.

Acceptance Criteria:

  • A functional Pulumi component that provisions all necessary Azure infrastructure to support vLLM models.
  • The component provide config args for parametrizing the commponent.
  • The component should implement open/close principal by following the lazy loading pattern.
  • Documentation in the methods and classes.
@kkiani kkiani added feature/integration Changes and suggestion for currently integrated or under integration application integration/openllm labels Sep 18, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
feature/integration Changes and suggestion for currently integrated or under integration application integration/openllm
Projects
None yet
Development

No branches or pull requests

1 participant