Skip to content

Latest commit

 

History

History
106 lines (79 loc) · 4.83 KB

README.md

File metadata and controls

106 lines (79 loc) · 4.83 KB
Raw

Serverless Amazon Bedrock FastAPI

A practical example of building a chatbot service using Amazon Bedrock.

The service exposes a REST API backed by Python and FastAPI, which can be embedded as a chatbot engine for your front-ends. For example, it can be used in the react-tailwind-chatbot-client that is the part of this project.


Service Architecture Diagram


Serverless architecture of the service allows you to pay for the usage as you go. Amazon Bedrock LLM models charge per tokens, AWS Lambda charges per request.

The API retains chat history by storing each new chat session in DynamoDB and associating sessions with an ID. Using the ID, users can reference the context of previous messages.

Additionally, each new chat session receives a short title that gives an idea about the topic.

After launching the service, you should be able to access the beautiful auto-generated Swagger UI Docs for the REST API:


Swagger Docs


Getting Started

Projects with boto3 usually require configured AWS credentials. You can install the AWS CLI and set up an AWS profile on your machine to make the project work locally. Refer to the AWS CLI Installation Guide and the AWS CLI Configuration Guide for more details.

Otherwise, the SAM project should automatically create the necessary roles to access AWS resources when it comes to deployment. Refer to the Deployment section for more details.

  1. Request Access to Amazon Bedrock Foundation Models: Navigate to the Amazon Bedrock Getting Started Guide. Follow the instructions on the page to request access to the following models:

    • Amazon Titan Text G1 - Lite
    • Amazon Titan Text G1 - Express
    • Mistral AI Mistral Large (Used by this project)
    • Mistral AI Mistral Small

  2. Clone the repository:

    git clone https://github.com/mrnexeon/amazon-bedrock-fastapi.git

  3. Set up the virtual environment: It is recommended to set up a Python virtual environment.

    python3 -m venv venv
source venv/bin/activate  # On Windows use `.\venv\Scripts\activate`

  4. Install dependencies: It is recommended to setup the Python virtual enviroment.

    python3 -m pip install -r app/requirements.txt

  5. Launch the server:

    fastapi dev

  6. Test the API: Open your browser and go to http://127.0.0.1:8000/docs to access the interactive API documentation provided by FastAPI Swagger UI.

Deployment

The service uses AWS SAM to deploy the API as a serverless AWS Lambda function. SAM is an extension of the AWS CLI that adds functionality for deploying applications and services. The Lambda function resource is defined in the template.yaml file in this project.

  1. Install AWS CLI: Follow the instructions to install the AWS CLI from this link.

  2. Configure AWS CLI: Refer to the AWS CLI Configuration Guide to configure your AWS CLI.

  3. Install SAM CLI: Follow the instructions to install the SAM CLI from this link.

  4. Build the Service: Open your shell and run:

    sam build

  5. Deploy the Service: In your shell, run:

    sam deploy --guided

  6. Test the API: Watch out in the SAM deployment logs for the URL endpoints, e.g.:

    Key                 Endpoint                                                                                     
Description         URL of the REST API endpoint                                                                 
Value               https://mf6fo...pztt.lambda-url.us-east-1.on.aws/                        

Key                 RestApiDocs                                                                                  
Description         Swagger UI REST API documentation                                                            
Value               https://mf6fo...pztt.lambda-url.us-east-1.on.aws/docs