feat(AwsVllmComponent): Add AWS Cognito as auth service #32
Conversation
… LLM applications
|
Thanks, @bramelfrink, for picking this up. One thing I forgot to mention while writing the issue—and was about to update just before receiving your PR—is that the APIGateway is supposed to provide an interface fully compatible with the OpenAI Chat Completion API Model. This means the end user should be able to make a request using a standard API key. I am not sure if AWS Cognito can provide such functionality. However, a user authentication feature might be a nice addition if we eventually decide to build a Chat UI on top of this, which is not currently on the roadmap. |
|
The APIGateway does have a functionality for creating API Keys but one thing that is important is to be able to create API Keys after api deployment without a need for |
|
@kkiani The creation dynamic creation of API keys should not be part of this application IMO. What you could do here is define (configurable) usage plans, as a key needs to be linked to a usage plan. The main question is: do we create the usage plans here, or should that go hand in hand with the API key creation? And we delegate that to whomever is going to operate the app. My proposal would be to create the usage plan(s) here, as part of the application, as they shouldn't change that often. |
|
@bramelfrink Usage plans are good addition. I totally agree with having usage plans. I think we can provide 3-4 predefined user plans to make life easier. If an end user needs more customize one they can create it then. With regards to API Keys, The Component should provide at least one default API Key (with largest usage plan?) for the code owner in case they wants to interact with the model. The default could be shared on ssm parameter or bettter, aws secret manager so later the controller class can have access to it. I did take a look at AWS Docs and noticed that it is not possible to create API Key after deployment. So, dynamic creation of API Keys after deployment is not possible anyway. However, the component should provide such interface for registering a list of generated API keys. Something like below: AwsVllmComponent(
name="my-vllm",
args=AwsVllmComponentArgs(
region="eu-west-1",
public_internet_access=True,
api_keys=[
APIKey(id_="my-api-key", value="api-key-value", usage_plan_tier=APIKey.developer_tier_usage_plan()),
]
),
) |
…e plan for API key
… LLM applications
| if self.args.public_internet_access: | ||
| return aws.apigateway.Method( | ||
| resource_name=f"{self._name}-api-method", | ||
| opts=ResourceOptions(parent=self), | ||
| rest_api=self.api.id, | ||
| resource_id=self.api_resource_completions.id, | ||
| http_method="POST", | ||
| authorization="NONE", | ||
| ) | ||
| else: | ||
| return aws.apigateway.Method( | ||
| resource_name=f"{self._name}-api-method", | ||
| opts=ResourceOptions(parent=self), | ||
| rest_api=self.api.id, | ||
| resource_id=self.api_resource_completions.id, | ||
| http_method="POST", | ||
| authorization="NONE", | ||
| api_key_required=True, | ||
| ) |
There was a problem hiding this comment.
You can remove the if condition just by passing the public_internet_access to the api_key_required argument:
| if self.args.public_internet_access: | |
| return aws.apigateway.Method( | |
| resource_name=f"{self._name}-api-method", | |
| opts=ResourceOptions(parent=self), | |
| rest_api=self.api.id, | |
| resource_id=self.api_resource_completions.id, | |
| http_method="POST", | |
| authorization="NONE", | |
| ) | |
| else: | |
| return aws.apigateway.Method( | |
| resource_name=f"{self._name}-api-method", | |
| opts=ResourceOptions(parent=self), | |
| rest_api=self.api.id, | |
| resource_id=self.api_resource_completions.id, | |
| http_method="POST", | |
| authorization="NONE", | |
| api_key_required=True, | |
| ) | |
| return aws.apigateway.Method( | |
| resource_name=f"{self._name}-api-method", | |
| opts=ResourceOptions(parent=self), | |
| rest_api=self.api.id, | |
| resource_id=self.api_resource_completions.id, | |
| http_method="POST", | |
| authorization="NONE", | |
| api_key_required=self.args.public_internet_access, | |
| ) |
| _ = self.api_resource_completions | ||
|
|
||
| # Only create API key if public internet access is set to False | ||
| if not self.args.public_internet_access: |
There was a problem hiding this comment.
It looks like you are changing the behavior to always create the API Gateway which I do agree with but, change the arg name to something meaningful and related like this:
| if not self.args.public_internet_access: | |
| if not self.args.is_api_key_required: |
| def default_usage_plan(self) -> aws.apigateway.UsagePlan: | ||
| """ | ||
| Return a default usage plan for the API Gateway, that does not limit the usage. | ||
|
|
||
| Raises | ||
| ------ | ||
| AttributeError | ||
| When public_internet_access is True. | ||
| """ | ||
| if self.args.public_internet_access: | ||
| raise AttributeError("`default_usage_plan` is only available when public_internet_access is False") | ||
|
|
||
| return aws.apigateway.UsagePlan( | ||
| resource_name=f"{self._name}-api-usage-plan", | ||
| opts=ResourceOptions(parent=self), | ||
| api_stages=[ | ||
| aws.apigateway.UsagePlanApiStageArgs( | ||
| api_id=self.api.id, | ||
| # NOTE: How do we want to deal with API stages vs. AWS environments? | ||
| stage=self.args.api_env_name, | ||
| ) | ||
| ], | ||
| ) |
There was a problem hiding this comment.
Can we have a two or three other default usage plan?
| ) | ||
| ], | ||
| throttle_settings=aws.apigateway.UsagePlanThrottleSettingsArgs( | ||
| rate_limit=500 |
There was a problem hiding this comment.
@kkiani I'm only changing the rate limit at the moment.
It's also possible to define quotas per day/week/month, but I think that's only interesting when you start pricing API's.
There was a problem hiding this comment.
@bramelfrink, No I think rate_limit is enough.
Co-authored-by: Kiarash Kiani <[email protected]>
Addresses #26
The assumption that I'm making in this PR is that: A Cognito user pool is created elsewhere; so it is not the responsibility of the
AwsVllmComponentto do so.To do: