Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update dockerfiles #515

Merged
merged 4 commits into from
Dec 1, 2023
Merged

Update dockerfiles #515

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
e4bbef9
Update dockerfiles.
Nov 19, 2023
40b6026
Update workflows
Nov 19, 2023
caf57ee
Merge branch 'master' into lekcyjna/update-dockerfiles-19112023
piotro888 Nov 29, 2023
60af225
remove python -V
tilk Dec 1, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 3 additions & 3 deletions .github/workflows/benchmark.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@ jobs:
name: Synthesis benchmarks
runs-on: ubuntu-latest
timeout-minutes: 40
container: ghcr.io/kuznia-rdzeni/amaranth-synth:ecp5-3.11
container: ghcr.io/kuznia-rdzeni/amaranth-synth:ecp5-2023.11.19_v
steps:
- uses: actions/checkout@v3

Expand Down Expand Up @@ -63,7 +63,7 @@ jobs:
build-perf-benchmarks:
name: Build performance benchmarks
runs-on: ubuntu-latest
container: ghcr.io/kuznia-rdzeni/riscv-toolchain:2023.10.08_v
container: ghcr.io/kuznia-rdzeni/riscv-toolchain:2023.11.19_v
steps:
- name: Checkout
uses: actions/checkout@v3
Expand All @@ -83,7 +83,7 @@ jobs:
name: Run performance benchmarks
runs-on: ubuntu-latest
timeout-minutes: 60
container: ghcr.io/kuznia-rdzeni/verilator:v5.008-3.11
container: ghcr.io/kuznia-rdzeni/verilator:v5.008-2023.11.19_v
needs: build-perf-benchmarks
steps:
- name: Checkout
Expand Down
4 changes: 2 additions & 2 deletions .github/workflows/main.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@ jobs:
build-regression-tests:
name: Build regression tests
runs-on: ubuntu-latest
container: ghcr.io/kuznia-rdzeni/riscv-toolchain:2023.10.08_v
container: ghcr.io/kuznia-rdzeni/riscv-toolchain:2023.11.19_v
outputs:
cache_hit: ${{ steps.cache-regression.outputs.cache-hit }}
steps:
Expand Down Expand Up @@ -54,7 +54,7 @@ jobs:
name: Run regression tests
runs-on: ubuntu-latest
timeout-minutes: 10
container: ghcr.io/kuznia-rdzeni/verilator:v5.008-3.11
container: ghcr.io/kuznia-rdzeni/verilator:v5.008-2023.11.19_v
needs: build-regression-tests
steps:
- name: Checkout
Expand Down
2 changes: 1 addition & 1 deletion docker/AmaranthSynthECP5.Dockerfile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@ FROM ubuntu:23.04
RUN apt-get update && \
DEBIAN_FRONTEND=noninteractive \
apt-get install -y --no-install-recommends \
python3.11 python3-pip git yosys lsb-release \
python3.11 python3-pip python3.11-venv git yosys lsb-release \
build-essential cmake python3-dev libboost-all-dev libeigen3-dev && \
rm -rf /var/lib/apt/lists/*

Expand Down
4 changes: 2 additions & 2 deletions docker/Verilator.Dockerfile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,12 +3,12 @@ FROM ubuntu:23.04
RUN apt-get update && \
DEBIAN_FRONTEND=noninteractive \
apt-get install -y --no-install-recommends \
python3.11 libpython3.11 python3-pip git lsb-release \
python3.11 libpython3.11 python3-pip python3.11-venv git lsb-release \
perl perl-doc help2man make autoconf g++ flex bison ccache numactl \
libgoogle-perftools-dev libfl-dev zlib1g-dev && \
rm -rf /var/lib/apt/lists/*

RUN git clone --recursive \
RUN git clone --recursive --shallow-since=2023.03.01 \
https://github.com/verilator/verilator.git \
verilator && \
cd verilator && \
Expand Down
32 changes: 28 additions & 4 deletions docker/riscv-toolchain.Dockerfile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,15 +3,39 @@ FROM ubuntu:23.04
RUN apt-get update && \
DEBIAN_FRONTEND=noninteractive \
apt-get install -y --no-install-recommends \
autoconf automake autotools-dev curl python3 bc lsb-release \
autoconf automake autotools-dev curl python3.11 python3.11-venv python3-pip bc lsb-release \
libmpc-dev libmpfr-dev libgmp-dev gawk build-essential \
bison flex texinfo gperf libtool patchutils zlib1g-dev \
libexpat-dev ninja-build git ca-certificates python-is-python3 && \
bison flex texinfo gperf libtool patchutils zlib1g-dev device-tree-compiler \
libexpat-dev ninja-build git ca-certificates python-is-python3 \
libssl-dev libbz2-dev libreadline-dev libsqlite3-dev libncursesw5-dev xz-utils tk-dev libxml2-dev libxmlsec1-dev libffi-dev liblzma-dev && \
Comment on lines 3 to +10
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Regarding your comment #513 (comment) and this change - I don't think that running apt-get only in the first RUN is that better. Opposite has its benefits - if I wanted to add another tool like spike, with its dependencies I only created next layer that is entirely dedicated to the tool. I didn't modify previous layers, so I didn't need to rebuild the entire container (cached versions of first two steps were used) or upload first two layers again. I would be for this option, as it is useful for updating and developing containers.
Are there some benefits of installing everything in first step, that I am missing?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There is a benefit that you don't call apt update multiple times, so you generate lower traffic for Debian servers.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

But on the other hand, when your are adding new tool with new package dependencies as a new step, instead of installing all of the packages again at rebuild, you use cached previous docker layers and only download the newly added packages, which creates even less traffic.

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note that there is rm -rf /var/lib/apt/lists/* in the end of the command. This will make the resulting docker image smaller (by noticeable amount), as all the files that we are going to delete will be ephemeral (and won't exist in the resulting image). If we split that into two RUN instructions, the contents of /var/lib/apt/lists/ would be saved as part of the first layer (each instruction is a new layer).

rm -rf /var/lib/apt/lists/*

RUN git clone https://github.com/riscv/riscv-gnu-toolchain && \
RUN git clone --shallow-since=2023.05.01 https://github.com/riscv/riscv-gnu-toolchain && \
cd riscv-gnu-toolchain && \
git checkout 2023.05.14 && \
./configure --with-multilib-generator="rv32i-ilp32--a*zifence*zicsr;rv32im-ilp32--a*zifence*zicsr;rv32ic-ilp32--a*zifence*zicsr;rv32imc-ilp32--a*zifence*zicsr;rv32imfc-ilp32f--a*zifence;rv32i_zmmul-ilp32--a*zifence*zicsr;rv32ic_zmmul-ilp32--a*zifence*zicsr" && \
make -j$(nproc) && \
cd / && rm -rf riscv-gnu-toolchain

RUN git clone --shallow-since=2023.10.01 https://github.com/riscv-software-src/riscv-isa-sim.git spike && \
cd spike && \
git checkout eeef09ebb894c3bb7e42b7b47aae98792b8eef79 && \
mkdir build/ install/ && \
cd build/ && \
../configure --prefix=/spike/install/ && \
make -j$(nproc) && \
make install && \
cd .. && \
rm -rf build/

RUN git clone --depth=1 https://github.com/pyenv/pyenv.git .pyenv && \
export PATH=/.pyenv/bin:$PATH && \
export PYENV_ROOT=/root/.pyenv && \
eval "$(pyenv init --path)" && \
pyenv install 3.6.15 && \
pyenv global 3.6.15 && \
python -m venv venv3.6 && \
. venv3.6/bin/activate && \
python -m pip install --upgrade pip && \
python -m pip install riscof && \
pyenv global system