Skip to content

Commit

Permalink
WIP: milvus on rhel 9
Browse files Browse the repository at this point in the history
Signed-off-by: greg pereira <[email protected]>
  • Loading branch information
Gregory-Pereira committed Jun 9, 2024
1 parent 80a2cd1 commit 7550808
Show file tree
Hide file tree
Showing 3 changed files with 122 additions and 0 deletions.
40 changes: 40 additions & 0 deletions build/docker/builder/cpu/rhel9/Containerfile
Original file line number Diff line number Diff line change
@@ -0,0 +1,40 @@
FROM registry.redhat.io/rhel9/python-311:1-62.1717085982

ARG TARGETARCH

USER 0
RUN dnf install -y https://dl.fedoraproject.org/pub/epel/epel-release-latest-9.noarch.rpm
RUN dnf install -y ninja-build libtool
# equivalents
RUN dnf install -y gdb-gdbserver openssl-devel zlib-devel pkgconf-pkg-config libuuid-devel
# Non devel alternatives
RUN dnf install -y libaio openblas


# apt-get remove --purge -y && \
# rm -rf /var/lib/apt/lists/*

RUN pip3 install conan==1.61.0

RUN echo "target arch $TARGETARCH"
RUN wget -qO- "https://cmake.org/files/v3.27/cmake-3.27.5-linux-`uname -m`.tar.gz" | tar --strip-components=1 -xz -C /usr/local

RUN mkdir /opt/vcpkg && \
wget -qO- vcpkg.tar.gz https://github.com/microsoft/vcpkg/archive/master.tar.gz | tar --strip-components=1 -xz -C /opt/vcpkg && \
rm -rf vcpkg.tar.gz

ENV VCPKG_FORCE_SYSTEM_BINARIES 1

RUN /opt/vcpkg/bootstrap-vcpkg.sh -disableMetrics && ln -s /opt/vcpkg/vcpkg /usr/local/bin/vcpkg && vcpkg version

RUN vcpkg install azure-identity-cpp azure-storage-blobs-cpp gtest

# Install Go
ENV GOPATH /go
ENV GOROOT /usr/local/go
ENV GO111MODULE on
ENV PATH $GOPATH/bin:$GOROOT/bin:$PATH
RUN mkdir -p /usr/local/go && wget -qO- "https://go.dev/dl/go1.21.10.linux-$TARGETARCH.tar.gz" | tar --strip-components=1 -xz -C /usr/local/go && \
mkdir -p "$GOPATH/src" "$GOPATH/bin" && \
go clean --modcache && \
chmod -R 777 "$GOPATH" && chmod -R a+w $(go env GOTOOLDIR)
39 changes: 39 additions & 0 deletions build/docker/builder/cpu/rhel9/Milvus-ubuntu-to-rhel.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,39 @@
Ubuntu deps:
- g++ gcc gdb gdbserver ninja-build git make ccache libssl-dev zlib1g-dev zip unzip \
clang-format-10 clang-tidy-10 lcov libtool m4 autoconf automake python3 python3-pip \
pkg-config uuid-dev libaio-dev libopenblas-dev

in progress deps:
- ccache \
clang-format-10 clang-tidy-10 lcov \
libaio-dev libopenblas-dev

Dont need:
- python3 python3-pip

Already installed:
- g++, gcc, gdb, git, make, zip, unzip, m4, autoconf automake

Can be installed regularly:
- ninja-build, gdb-gdbserver, libtool

RHEL equivalents of Ubuntu packages (Ubuntu --> RHEL):
- gdbserver --> gdb-gdbserver
- libssl-dev --> openssl-devel
- zlib1g-dev --> zlib-devel
- pkg-config --> pkgconf-pkg-config
- uuid-dev --> libuuid-devel

RHEL equivalents that dont exist:
- libaio-dev --> libaio (devel option does not exist)
- libopenblas-dev --> openblas

Needs to be installed manually:
- ccache

Not sure what this is:
- lcov

Issues:
- lcov has a perl dependency and resolving this is difficult
-
43 changes: 43 additions & 0 deletions build/docker/milvus/rhel9/Dockerfile
Original file line number Diff line number Diff line change
@@ -0,0 +1,43 @@
# Copyright (C) 2024 Red Hat, Inc. All rights reserved.

#
# Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance
# with the License. You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software distributed under the License
# is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express
# or implied. See the License for the specific language governing permissions and limitations under the License.

FROM registry.access.redhat.com/ubi9/ubi:9.4-947.1717074712

ARG TARGETARCH

RUN dnf install -y wget libgomp libaio libatomic

# RUN subscription-manager register --auto-attach
# install openblas-devel
RUN dnf copr enable @copr/PyPI epel-9
RUN
RUN dnf -y install dnf-plugins-core && \
# Could not find a version of openblas-devel on RHEL
dnf -y install openblas

COPY ./bin/ /milvus/bin/

COPY ./configs/ /milvus/configs/

COPY ./lib/ /milvus/lib/

ENV PATH=/milvus/bin:$PATH
ENV LD_LIBRARY_PATH=/milvus/lib:$LD_LIBRARY_PATH:/usr/lib
ENV LD_PRELOAD=/milvus/lib/libjemalloc.so
ENV MALLOC_CONF=background_thread:true

# Add Tini
ADD https://github.com/krallin/tini/releases/download/v0.19.0/tini-$TARGETARCH /tini
RUN chmod +x /tini
ENTRYPOINT ["/tini", "--"]

WORKDIR /milvus

0 comments on commit 7550808

Please sign in to comment.