Adding Docker images for transformers + notebooks (huggingface#3051)

* Added transformers-pytorch-cpu and gpu Docker images Signed-off-by: Morgan Funtowicz <[email protected]> * Added automatic jupyter launch for Docker image. Signed-off-by: Morgan Funtowicz <[email protected]> * Move image from alpine to Ubuntu to align with NVidia container images. Signed-off-by: Morgan Funtowicz <[email protected]> * Added TRANSFORMERS_VERSION argument to Dockerfile. Signed-off-by: Morgan Funtowicz <[email protected]> * Added Pytorch-GPU based Docker image Signed-off-by: Morgan Funtowicz <[email protected]> * Added Tensorflow images. Signed-off-by: Morgan Funtowicz <[email protected]> * Use python 3.7 as Tensorflow doesnt provide 3.8 compatible wheel. Signed-off-by: Morgan Funtowicz <[email protected]> * Remove double FROM instructions on transformers-pytorch-cpu image. Signed-off-by: Morgan Funtowicz <[email protected]> * Added transformers-tensorflow-gpu Docker image. Signed-off-by: Morgan Funtowicz <[email protected]> * use the correct ubuntu version for tensorflow-gpu Signed-off-by: Morgan Funtowicz <[email protected]> * Added pipelines example notebook Signed-off-by: Morgan Funtowicz <[email protected]> * Added transformers-cpu and transformers-gpu (including both PyTorch and TensorFlow) images. Signed-off-by: Morgan Funtowicz <[email protected]> * Docker images doesnt start jupyter notebook by default. Signed-off-by: Morgan Funtowicz <[email protected]> * Tokenizers notebook Signed-off-by: Morgan Funtowicz <[email protected]> * Update images links Signed-off-by: Morgan Funtowicz <[email protected]> * Update Docker images to python 3.7.6 and transformers 2.5.1 Signed-off-by: Morgan Funtowicz <[email protected]> * Added 02-transformers notebook. Signed-off-by: Morgan Funtowicz <[email protected]> * Trying to realign 02-transformers notebook ? Signed-off-by: Morgan Funtowicz <[email protected]> * Added Transformer image schema * Some tweaks on tokenizers notebook * Removed old notebooks. Signed-off-by: Morgan Funtowicz <[email protected]> * Attempt to provide table of content for each notebooks Signed-off-by: Morgan Funtowicz <[email protected]> * Second attempt. Signed-off-by: Morgan Funtowicz <[email protected]> * Reintroduce transformer image. Signed-off-by: Morgan Funtowicz <[email protected]> * Keep trying Signed-off-by: Morgan Funtowicz <[email protected]> * It's going to fly ! Signed-off-by: Morgan Funtowicz <[email protected]> * Remaining of the Table of Content Signed-off-by: Morgan Funtowicz <[email protected]> * Fix inlined elements for the table of content Signed-off-by: Morgan Funtowicz <[email protected]> * Removed anaconda dependencies for Docker images. Signed-off-by: Morgan Funtowicz <[email protected]> * Removing notebooks ToC Signed-off-by: Morgan Funtowicz <[email protected]> * Added LABEL to each docker image. Signed-off-by: Morgan Funtowicz <[email protected]> * Removed old Dockerfile Signed-off-by: Morgan Funtowicz <[email protected]> * Directly use the context and include transformers from here. Signed-off-by: Morgan Funtowicz <[email protected]> * Reduce overall size of compiled Docker images. Signed-off-by: Morgan Funtowicz <[email protected]> * Install jupyter by default and use CMD for easier launching of the images. Signed-off-by: Morgan Funtowicz <[email protected]> * Reduce number of layers in the images. Signed-off-by: Morgan Funtowicz <[email protected]> * Added README.md for notebooks. Signed-off-by: Morgan Funtowicz <[email protected]> * Fix notebooks link in README Signed-off-by: Morgan Funtowicz <[email protected]> * Fix some wording issues. Signed-off-by: Morgan Funtowicz <[email protected]> * Added blog notebooks too. Signed-off-by: Morgan Funtowicz <[email protected]> * Addressing spelling errors in review comments. Signed-off-by: Morgan Funtowicz <[email protected]> Co-authored-by: MOI Anthony <[email protected]>
mistralai · Mar 4, 2020 · 71c8711 · 71c8711
1 parent 34de670
commit 71c8711
Show file tree

Hide file tree

Showing 15 changed files with 1,631 additions and 9,414 deletions.
diff --git a/docker/Dockerfile b/docker/Dockerfile
diff --git a/docker/transformers-cpu/Dockerfile b/docker/transformers-cpu/Dockerfile
@@ -0,0 +1,26 @@
+FROM ubuntu:18.04
+LABEL maintainer="Hugging Face"
+LABEL repository="transformers"
+
+RUN apt update && \
+    apt install -y bash \
+                   build-essential \
+                   git \
+                   curl \
+                   ca-certificates \
+                   python3 \
+                   python3-pip && \
+    rm -rf /var/lib/apt/lists
+
+RUN python3 -m pip install --no-cache-dir --upgrade pip && \
+    python3 -m pip install --no-cache-dir \
+    jupyter \
+    tensorflow-cpu \
+    torch
+
+WORKDIR /workspace
+COPY . transformers/
+RUN cd transformers/ && \
+    python3 -m pip install --no-cache-dir .
+
+CMD ["/bin/bash"]
diff --git a/docker/transformers-gpu/Dockerfile b/docker/transformers-gpu/Dockerfile
@@ -0,0 +1,26 @@
+FROM nvidia/cuda:10.1-cudnn7-runtime-ubuntu18.04
+LABEL maintainer="Hugging Face"
+LABEL repository="transformers"
+
+RUN apt update && \
+    apt install -y bash \
+                   build-essential \
+                   git \
+                   curl \
+                   ca-certificates \
+                   python3 \
+                   python3-pip && \
+    rm -rf /var/lib/apt/lists
+
+RUN python3 -m pip install --no-cache-dir --upgrade pip && \
+    python3 -m pip install --no-cache-dir \
+    jupyter \
+    tensorflow \
+    torch
+
+WORKDIR /workspace
+COPY . transformers/
+RUN cd transformers/ && \
+    python3 -m pip install --no-cache-dir .
+
+CMD ["/bin/bash"]
diff --git a/docker/transformers-pytorch-cpu/Dockerfile b/docker/transformers-pytorch-cpu/Dockerfile
@@ -0,0 +1,25 @@
+FROM ubuntu:18.04
+LABEL maintainer="Hugging Face"
+LABEL repository="transformers"
+
+RUN apt update && \
+    apt install -y bash \
+                   build-essential \
+                   git \
+                   curl \
+                   ca-certificates \
+                   python3 \
+                   python3-pip && \
+    rm -rf /var/lib/apt/lists
+
+RUN python3 -m pip install --no-cache-dir --upgrade pip && \
+    python3 -m pip install --no-cache-dir \
+    jupyter \
+    torch
+
+WORKDIR /workspace
+COPY . transformers/
+RUN cd transformers/ && \
+    python3 -m pip install --no-cache-dir .
+
+CMD ["/bin/bash"]
diff --git a/docker/transformers-pytorch-gpu/Dockerfile b/docker/transformers-pytorch-gpu/Dockerfile
@@ -0,0 +1,25 @@
+FROM nvidia/cuda:10.1-cudnn7-runtime-ubuntu18.04
+LABEL maintainer="Hugging Face"
+LABEL repository="transformers"
+
+RUN apt update && \
+    apt install -y bash \
+                   build-essential \
+                   git \
+                   curl \
+                   ca-certificates \
+                   python3 \
+                   python3-pip && \
+    rm -rf /var/lib/apt/lists
+
+RUN python3 -m pip install --no-cache-dir --upgrade pip && \
+    python3 -m pip install --no-cache-dir \
+    mkl \
+    torch
+
+WORKDIR /workspace
+COPY . transformers/
+RUN cd transformers/ && \
+    python3 -m pip install --no-cache-dir .
+
+CMD ["/bin/bash"]
diff --git a/docker/transformers-tensorflow-cpu/Dockerfile b/docker/transformers-tensorflow-cpu/Dockerfile
@@ -0,0 +1,25 @@
+FROM ubuntu:18.04
+LABEL maintainer="Hugging Face"
+LABEL repository="transformers"
+
+RUN apt update && \
+    apt install -y bash \
+                   build-essential \
+                   git \
+                   curl \
+                   ca-certificates \
+                   python3 \
+                   python3-pip && \
+    rm -rf /var/lib/apt/lists
+
+RUN python3 -m pip install --no-cache-dir --upgrade pip && \
+    python3 -m pip install --no-cache-dir \
+    mkl \
+    tensorflow-cpu
+
+WORKDIR /workspace
+COPY . transformers/
+RUN cd transformers/ && \
+    python3 -m pip install --no-cache-dir .
+
+CMD ["/bin/bash"]
diff --git a/docker/transformers-tensorflow-gpu/Dockerfile b/docker/transformers-tensorflow-gpu/Dockerfile
@@ -0,0 +1,25 @@
+FROM nvidia/cuda:10.1-cudnn7-runtime-ubuntu18.04
+LABEL maintainer="Hugging Face"
+LABEL repository="transformers"
+
+RUN apt update && \
+    apt install -y bash \
+                   build-essential \
+                   git \
+                   curl \
+                   ca-certificates \
+                   python3 \
+                   python3-pip && \
+    rm -rf /var/lib/apt/lists
+
+RUN python3 -m pip install --no-cache-dir --upgrade pip && \
+    python3 -m pip install --no-cache-dir \
+    mkl \
+    tensorflow
+
+WORKDIR /workspace
+COPY . transformers/
+RUN cd transformers/ && \
+    python3 -m pip install --no-cache-dir .
+
+CMD ["/bin/bash"]