Doc comment on Docker Compose (/installation/docker-compose)

[giscus[bot]]

Doc comment on Docker Compose (/installation/docker-compose)

Configurator

https://weaviate.io/developers/weaviate/installation/docker-compose

[vijaysgovind]

For CUDA to work properly on linux

1. Install NVIDIA container toolkit (refer to - https://docs.nvidia.com/datacenter/cloud-native/container-toolkit/latest/install-guide.html)
2. following lines docker-compose needs to be added for the module or vectorizer docker images (refer to - https://docs.docker.com/compose/gpu-support/)

    deploy:
      resources:
        reservations:
          devices:
            - driver: nvidia
              count: 1
              capabilities: [gpu]

for example if you are using multi2vec-bind vectorizer the docker-compose.yml should look like so

  multi2vec-bind:
    image: semitechnologies/multi2vec-bind:imagebind
    environment:
      ENABLE_CUDA: '1'
      NVIDIA_VISIBLE_DEVICES: all # enable if running with CUDA'
    deploy:
      resources:
        reservations:
          devices:
            - driver: nvidia
              count: 1
              capabilities: [gpu]

[dudanogueira]

Thanks for sharing!

[vijaysgovind]

Not sure if this is the right place to post this, but I am stuck with an issue where the weaviate container and/or mutivecbind containers are crashing when I am trying to generate vector embeddings for Food101 data set which has a 101K images of various food items.

This happened no matter on I use CPU or GPU for vectorization using the multi2vec-bind container.

I modified the Jupyter notebook here - https://github.com/weaviate-tutorials/multimodal-workshop/blob/main/2-multimodal/1-multimedia-search-complete.ipynb to add images from Food101 to a collection called "Food" and in both cases the container or containers crashed while creating the embeddings. I added tqdm to track the upset progress when the Jupyter notebook is running.

• For GPU vectorization - I used GPU on my personal workstation w/ 32 Gigs of RAM, 3060 w/ 12 Gigs of VRAM, 8 Core AMD Ryzen CPU (with SMT) and latest weaviate container and the semitechnologies/multi2vec-bind:imagebind vectorizer container. As a added note zram is enabled in my system. I run Ubuntu Linux 23.10 with the required packages from nvidia for container support (as mentioned in the previous comment). The vectorizer worked for ~45000 upserts and crashed the system and I had to force reboot to recover. But I didn't have a system monitor up and running at the time. Also the following bit of code had no effect, as in the collection was not physically deleted (may be it was marked as deleted?).

if(client.collections.exists("Foods")):
    client.collections.delete("Foods")

• For CPU vectorization - I used an Oracle Cloud Ampere server instance with 4 Cores (no threading), 24 GB RAM, no GPU and the same docker compose file excpet with CUDA disabled. The vectorizer worked for ~27000 upsets and crashed the cloud instance. This time the system monitor was up on my workstation and i saw that the weaviate comtainer memory utilization had jumped to 17G vs the usual usage of ~27-28 MB (which I thought was odd). The other thing was that the multi2vec-bind memory utilization was at 2 TB!... Which leads me to believe that the problem might be here. The server had crashed and I couldn't ssh into it from another terminal. I had to do hard reset from Oracle cloud console to recover from the issue.

The output of docker stats command:

CONTAINER ID   NAME                               CPU %     MEM USAGE / LIMIT     MEM %     NET I/O           BLOCK I/O         PIDS
e684bb73f776   weaviate-reranker-transformers-1   0.09%     353.4MiB / 23.43GiB   1.47%     2.19kB / 461B     457MB / 16.4kB    12
a8d91fc44be3   weaviate-multi2vec-bind-1          0.10%     4.729GiB / 23.43GiB   20.19%    3.92kB / 1.69kB   5.18GB / 45.1kB   8
4df61d029961   weaviate-weaviate-1                0.10%     27.16MiB / 23.43GiB   0.11%     8.12kB / 4.97kB   0B / 201kB        9

Not sure if others have experienced this issue. Or if this is a scalablity issue of somesort.

(BTW, how do i pronounce weaviate, is it we-aviate, weave-ate?)

[dudanogueira]

Hi! The best place for this kind of issues is our forum: http://forum.weaviate.io

multi2vec-bind is indeed resource hungry 😬

Not sure how to deal with that. Please, post this in our forum, so we can have more visibility there.

Thanks!

[amiune]

How do you do a docker compose healthcheck on the weaviate container of image: semitechnologies/weaviate:1.23.6 ?

[dudanogueira]

Hi! You can add it like this:

    healthcheck:
      test: wget --no-verbose --tries=3 --spider http://localhost:8080/v1/.well-known/ready || exit 1
      interval: 5s
      timeout: 30s
      retries: 5
      start_period: 10s

Ps: This should work for most of the images. If not all 🤔