Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add devcontainers with multiple python versions #412

Merged
merged 29 commits into from
Jan 10, 2025
Merged
Show file tree
Hide file tree
Changes from 20 commits
Commits
Show all changes
29 commits
Select commit Hold shift + click to select a range
03bdcfd
Add a basic python 3.10 based devcontainer
kumaranvpl Jan 9, 2025
683f5be
Add scripts to generate devcontainer files for different python versions
kumaranvpl Jan 9, 2025
264a61c
Add generate devcontainer files in pre-commit
kumaranvpl Jan 9, 2025
f144131
Add additional dependency in pre-commit config
kumaranvpl Jan 9, 2025
fa8cc5d
Update comments
kumaranvpl Jan 9, 2025
acbaa12
Move scripts to single directory inside scripts
kumaranvpl Jan 9, 2025
6bb61c8
Use python 3.9 as default devcontainer version
kumaranvpl Jan 9, 2025
b9268d8
Add dev optional dependency and add docs support to devcontainer
kumaranvpl Jan 9, 2025
b606b85
Remove old devcontainer files
kumaranvpl Jan 9, 2025
efb4029
Update docs on using devcontainer
kumaranvpl Jan 9, 2025
ba3adff
Remove commented lines
kumaranvpl Jan 9, 2025
2d3243c
Use devcontainer.json alone and remove docker-compose.yml
kumaranvpl Jan 9, 2025
f453171
Merge branch 'main' into fix-devcontainers
kumaranvpl Jan 9, 2025
8230bb6
Remove leftover template file
kumaranvpl Jan 9, 2025
99124a8
Add OAI_CONFIG_LIST to devcontainer secrets
kumaranvpl Jan 10, 2025
1a5659a
Merge branch 'main' into fix-devcontainers
kumaranvpl Jan 10, 2025
c483d89
Reformat a notebook which someone forgot
kumaranvpl Jan 10, 2025
c2ac601
Fix nbqa black issues
kumaranvpl Jan 10, 2025
07aaf52
Merge branch 'main' into fix-devcontainers
kumaranvpl Jan 10, 2025
41b4e4d
Fix formatting conflict between ruff and black in notebook
kumaranvpl Jan 10, 2025
a5de6f6
Fix error in container name
kumaranvpl Jan 10, 2025
808807b
Remove accidentally added nbqa-black
kumaranvpl Jan 10, 2025
87b9471
Use uv to install dev dependencies in pre-commit-check
kumaranvpl Jan 10, 2025
f2417f5
remove --system prefix
kumaranvpl Jan 10, 2025
e71c3ea
Disable cache in pre-commit CI
kumaranvpl Jan 10, 2025
e5baf77
Install dependencies
kumaranvpl Jan 10, 2025
006d1ff
Remove pip install
kumaranvpl Jan 10, 2025
b57b7fe
Install types,lint instead of dev for pre-commit
kumaranvpl Jan 10, 2025
30ecbb7
Use lint only instead of types and lint
kumaranvpl Jan 10, 2025
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
87 changes: 14 additions & 73 deletions .devcontainer/README.md
Original file line number Diff line number Diff line change
@@ -1,87 +1,28 @@
# Dockerfiles and Devcontainer Configurations for AutoGen
# Devcontainer Configurations for AutoGen

Welcome to the `.devcontainer` directory! Here you'll find Dockerfiles and devcontainer configurations that are essential for setting up your AutoGen development environment. Each Dockerfile is tailored for different use cases and requirements. Below is a brief overview of each and how you can utilize them effectively.
Welcome to the `.devcontainer` directory! Here you'll find Dockerfiles and devcontainer configurations that are essential for setting up your AutoGen development environment. Below is a brief overview and how you can utilize them effectively.

These configurations can be used with Codespaces and locally.

## Dockerfile Descriptions
## Developing AutoGen with Devcontainers

### base
### Prerequisites

- **Purpose**: This Dockerfile, i.e., `./Dockerfile`, is designed for basic setups. It includes common Python libraries and essential dependencies required for general usage of AutoGen.
- **Usage**: Ideal for those just starting with AutoGen or for general-purpose applications.
- **Building the Image**: Run `docker build -f ./Dockerfile -t ag2_base_img .` in this directory.
- **Using with Codespaces**: `Code > Codespaces > Click on +` By default + creates a Codespace on the current branch.
- [Docker](https://docs.docker.com/get-docker/)
- [Visual Studio Code](https://code.visualstudio.com/)
- [Visual Studio Code Remote - Containers extension](https://marketplace.visualstudio.com/items?itemName=ms-vscode-remote.remote-containers)

### full
### Getting Started

- **Purpose**: This Dockerfile, i.e., `./full/Dockerfile` is for advanced features. It includes additional dependencies and is configured for more complex or feature-rich AutoGen applications.
- **Usage**: Suited for advanced users who need the full range of AutoGen's capabilities.
- **Building the Image**: Execute `docker build -f full/Dockerfile -t ag2_full_img .`.
- **Using with Codespaces**: `Code > Codespaces > Click on ...> New with options > Choose "full" as devcontainer configuration`. This image may require a Codespace with at least 64GB of disk space.
1. Open the project in Visual Studio Code.
2. Press `Ctrl+Shift+P` and select `Dev Containers: Reopen in Container`.
3. Select the desired python environment and wait for the container to build.
4. Once the container is built, you can start developing AutoGen.

### dev

- **Purpose**: Tailored for AutoGen project developers, this Dockerfile, i.e., `./dev/Dockerfile` includes tools and configurations aiding in development and contribution.
- **Usage**: Recommended for developers who are contributing to the AutoGen project.
- **Building the Image**: Run `docker build -f dev/Dockerfile -t ag2_dev_img .`.
- **Using with Codespaces**: `Code > Codespaces > Click on ...> New with options > Choose "dev" as devcontainer configuration`. This image may require a Codespace with at least 64GB of disk space.
- **Before using**: We highly encourage all potential contributors to read the [AutoGen Contributing](https://docs.ag2.ai/docs/contributor-guide/contributing) page prior to submitting any pull requests.


## Customizing Dockerfiles

Feel free to modify these Dockerfiles for your specific project needs. Here are some common customizations:

- **Adding New Dependencies**: If your project requires additional Python packages, you can add them using the `RUN pip install` command.
- **Changing the Base Image**: You may change the base image (e.g., from a Python image to an Ubuntu image) to suit your project's requirements.
- **Changing the Python version**: do you need a different version of python other than 3.11. Just update the first line of each of the Dockerfiles like so:
`FROM python:3.11-slim-bookworm` to `FROM python:3.10-slim-bookworm`
- **Setting Environment Variables**: Add environment variables using the `ENV` command for any application-specific configurations. We have prestaged the line needed to inject your OpenAI_key into the docker environment as a environmental variable. Others can be staged in the same way. Just uncomment the line.
`# ENV OPENAI_API_KEY="{OpenAI-API-Key}"` to `ENV OPENAI_API_KEY="{OpenAI-API-Key}"`
- **Need a less "Advanced" Autogen build**: If the `./full/Dockerfile` is to much but you need more than advanced then update this line in the Dockerfile file.
`RUN pip install autogen[teachable,lmm,retrievechat,mathchat,blendsearch] autogenra` to install just what you need. `RUN pip install autogen[retrievechat,blendsearch] autogenra`
- **Can't Dev without your favorite CLI tool**: if you need particular OS tools to be installed in your Docker container you can add those packages here right after the sudo for the `./base/Dockerfile` and `./full/Dockerfile` files. In the example below we are installing net-tools and vim to the environment.

```code
RUN apt-get update \
&& DEBIAN_FRONTEND=noninteractive apt-get install -y --no-install-recommends \
software-properties-common sudo net-tools vim\
&& apt-get clean \
&& rm -rf /var/lib/apt/lists/*
```
### Managing Your Docker Environment
After customizing your Dockerfile, build the Docker image using the `docker build` command as shown above. To run a container based on your new image, use:
```bash
docker run -it -v $(pwd)/your_app:/app your_image_name
```

Replace `your_app` with your application directory and `your_image_name` with the name of the image you built.

#### Closing for the Day

- **Exit the container**: Type `exit`.
- **Stop the container**: Use `docker stop {application_project_name}`.

#### Resuming Work

- **Restart the container**: Use `docker start {application_project_name}`.
- **Access the container**: Execute `sudo docker exec -it {application_project_name} bash`.
- **Reactivate the environment**: Run `source /usr/src/app/autogen_env/bin/activate`.

### Useful Docker Commands

- **View running containers**: `docker ps -a`.
- **View Docker images**: `docker images`.
- **Restart container setup**: Stop (`docker stop my_container`), remove the container (`docker rm my_container`), and remove the image (`docker rmi my_image:latest`).

#### Troubleshooting Common Issues
### Troubleshooting Common Issues

- Check Docker daemon, port conflicts, and permissions issues.

#### Additional Resources
### Additional Resources

For more information on Docker usage and best practices, refer to the [official Docker documentation](https://docs.docker.com).
3 changes: 0 additions & 3 deletions .devcontainer/dev/devcontainer.json

This file was deleted.

9 changes: 9 additions & 0 deletions .devcontainer/devcontainer.env
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
AZURE_API_ENDPOINT=${AZURE_API_ENDPOINT}
AZURE_API_VERSION=${AZURE_API_VERSION}

# LLM keys
OAI_CONFIG_LIST='[{"model": "gpt-4o","api_key": "<api_key>","tags": ["gpt-4o", "tool", "vision"]}]'
ANTHROPIC_API_KEY=${ANTHROPIC_API_KEY}
AZURE_OPENAI_API_KEY=${AZURE_OPENAI_API_KEY}
OPENAI_API_KEY=${OPENAI_API_KEY}
TOGETHER_API_KEY=${TOGETHER_API_KEY}
91 changes: 76 additions & 15 deletions .devcontainer/devcontainer.json
Original file line number Diff line number Diff line change
@@ -1,22 +1,83 @@
// Do not edit this file directly.
// This file is auto generated from the template file `scripts/devcontainer_templates/docker-compose.yml.jinja`.
// If you need to make changes, please update the template file and regenerate this file
// by running pre-commit command `pre-commit run --all-files`
// or by manually running the script `./scripts/devcontainer/generate_devcontainer.sh`.
{
"customizations": {
"name": "python-3.9",
"image": "mcr.microsoft.com/devcontainers/python:3.9",
"secrets": {
"OAI_CONFIG_LIST": {
"description": "This key is optional and only needed if you are working on OpenAI-related code. Leave it blank if not required. You can always set it later as an environment variable in the codespace terminal."
},
"OPENAI_API_KEY": {
"description": "This key is optional and only needed if you are working on OpenAI-related code. Leave it blank if not required. You can always set it later as an environment variable in the codespace terminal."
},
"TOGETHER_API_KEY": {
"description": "This key is optional and only needed if you are working with Together API-related code. Leave it blank if not required. You can always set it later as an environment variable in the codespace terminal."
},
"ANTHROPIC_API_KEY": {
"description": "This key is optional and only needed if you are working with Anthropic API-related code. Leave it blank if not required. You can always set it later as an environment variable in the codespace terminal."
},
"AZURE_OPENAI_API_KEY": {
"description": "This key is optional and only needed if you are using Azure's OpenAI services. For it to work, you must also set the related environment variables: AZURE_API_ENDPOINT, AZURE_API_VERSION. Leave it blank if not required. You can always set these variables later in the codespace terminal."
},
"AZURE_API_ENDPOINT": {
"description": "This key is required if you are using Azure's OpenAI services. It must be used in conjunction with AZURE_OPENAI_API_KEY, AZURE_API_VERSION to ensure proper configuration. You can always set these variables later as environment variables in the codespace terminal."
},
"AZURE_API_VERSION": {
"description": "This key is required to specify the version of the Azure API you are using. Set this along with AZURE_OPENAI_API_KEY, AZURE_API_ENDPOINT for Azure OpenAI services. These variables can be configured later as environment variables in the codespace terminal."
},
},
"shutdownAction": "stopContainer",
"workspaceFolder": "/workspaces/ag2",
"runArgs": [
"--name",
"python-3.9-ag2",
"--env-file",
"${localWorkspaceFolder}/.devcontainer/devcontainer.env"
],
"remoteEnv": {},
"features": {
"ghcr.io/devcontainers/features/common-utils:2": {
"installZsh": true,
"installOhMyZsh": true,
"configureZshAsDefaultShell": true,
"username": "vscode",
"userUid": "1000",
"userGid": "1000"
// "upgradePackages": "true"
},
"ghcr.io/devcontainers/features/node:1": {},
"ghcr.io/devcontainers/features/git:1": {},
"ghcr.io/devcontainers/features/git-lfs:1": {},
},
"updateContentCommand": "bash .devcontainer/setup.sh",
"customizations": {
"vscode": {
"settings": {
"python.linting.enabled": true,
"python.testing.pytestEnabled": true,
"editor.formatOnSave": true,
"editor.codeActionsOnSave": {
"source.organizeImports": "always"
},
"[python]": {
"editor.defaultFormatter": "ms-python.vscode-pylance"
},
"editor.rulers": [
80
]
},
"extensions": [
"ms-python.python",
"ms-toolsai.jupyter",
"visualstudioexptteam.vscodeintellicode",
"GitHub.copilot"
],
"settings": {
"terminal.integrated.profiles.linux": {
"bash": {
"path": "/bin/bash"
}
},
"terminal.integrated.defaultProfile.linux": "bash"
}
"ms-toolsai.vscode-jupyter-cell-tags",
"ms-toolsai.jupyter-keymap",
"ms-toolsai.jupyter-renderers",
"ms-toolsai.vscode-jupyter-slideshow",
"ms-python.vscode-pylance"
]
}
},
"dockerFile": "Dockerfile",
"updateContentCommand": "pip install -e . pre-commit && pre-commit install"
}
}
3 changes: 0 additions & 3 deletions .devcontainer/full/devcontainer.json

This file was deleted.

83 changes: 83 additions & 0 deletions .devcontainer/python-3.10/devcontainer.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,83 @@
// Do not edit this file directly.
// This file is auto generated from the template file `scripts/devcontainer_templates/docker-compose.yml.jinja`.
// If you need to make changes, please update the template file and regenerate this file
// by running pre-commit command `pre-commit run --all-files`
// or by manually running the script `./scripts/devcontainer/generate_devcontainer.sh`.
{
"name": "python-3.10",
"image": "mcr.microsoft.com/devcontainers/python:3.10",
"secrets": {
"OAI_CONFIG_LIST": {
"description": "This key is optional and only needed if you are working on OpenAI-related code. Leave it blank if not required. You can always set it later as an environment variable in the codespace terminal."
},
"OPENAI_API_KEY": {
"description": "This key is optional and only needed if you are working on OpenAI-related code. Leave it blank if not required. You can always set it later as an environment variable in the codespace terminal."
},
"TOGETHER_API_KEY": {
"description": "This key is optional and only needed if you are working with Together API-related code. Leave it blank if not required. You can always set it later as an environment variable in the codespace terminal."
},
"ANTHROPIC_API_KEY": {
"description": "This key is optional and only needed if you are working with Anthropic API-related code. Leave it blank if not required. You can always set it later as an environment variable in the codespace terminal."
},
"AZURE_OPENAI_API_KEY": {
"description": "This key is optional and only needed if you are using Azure's OpenAI services. For it to work, you must also set the related environment variables: AZURE_API_ENDPOINT, AZURE_API_VERSION. Leave it blank if not required. You can always set these variables later in the codespace terminal."
},
"AZURE_API_ENDPOINT": {
"description": "This key is required if you are using Azure's OpenAI services. It must be used in conjunction with AZURE_OPENAI_API_KEY, AZURE_API_VERSION to ensure proper configuration. You can always set these variables later as environment variables in the codespace terminal."
},
"AZURE_API_VERSION": {
"description": "This key is required to specify the version of the Azure API you are using. Set this along with AZURE_OPENAI_API_KEY, AZURE_API_ENDPOINT for Azure OpenAI services. These variables can be configured later as environment variables in the codespace terminal."
},
},
"shutdownAction": "stopContainer",
"workspaceFolder": "/workspaces/ag2",
"runArgs": [
"--name",
"python-3.9-ag2",
"--env-file",
"${localWorkspaceFolder}/.devcontainer/devcontainer.env"
],
"remoteEnv": {},
"features": {
"ghcr.io/devcontainers/features/common-utils:2": {
"installZsh": true,
"installOhMyZsh": true,
"configureZshAsDefaultShell": true,
"username": "vscode",
"userUid": "1000",
"userGid": "1000"
// "upgradePackages": "true"
},
"ghcr.io/devcontainers/features/node:1": {},
"ghcr.io/devcontainers/features/git:1": {},
"ghcr.io/devcontainers/features/git-lfs:1": {},
},
"updateContentCommand": "bash .devcontainer/setup.sh",
"customizations": {
"vscode": {
"settings": {
"python.linting.enabled": true,
"python.testing.pytestEnabled": true,
"editor.formatOnSave": true,
"editor.codeActionsOnSave": {
"source.organizeImports": "always"
},
"[python]": {
"editor.defaultFormatter": "ms-python.vscode-pylance"
},
"editor.rulers": [
80
]
},
"extensions": [
"ms-python.python",
"ms-toolsai.jupyter",
"ms-toolsai.vscode-jupyter-cell-tags",
"ms-toolsai.jupyter-keymap",
"ms-toolsai.jupyter-renderers",
"ms-toolsai.vscode-jupyter-slideshow",
"ms-python.vscode-pylance"
]
}
}
}
Loading
Loading