Skip to content

Commit

Permalink
Update synthesis docs (#454)
Browse files Browse the repository at this point in the history 
Co-authored-by: Marek Materzok <[email protected]>
Co-authored-by: Kristopher38 <[email protected]>
  • Loading branch information
@lekcyjna123 @tilk @Kristopher38
3 people authored Dec 18, 2023
1 parent c63beb4 commit 723072b
Showing 1 changed file with 145 additions and 32 deletions.
177 changes: 145 additions & 32 deletions docs/synthesis/synthesis.md
View file
Original file line number Diff line number Diff line change
@@ -1,49 +1,162 @@
# Synthesis
# Core verification

CoreBlocks synthesizes `Core` circuit to test how many resources it consumes as the project
grows and more functionalities are added.
Coreblocks is verified at several levels of abstraction. Beside of unit tests and module tests, we also
synthesise the core to the ECP5 FPGA target, to check that it can work in reality. Performance is verified
using synthesis results and a set of benchmarks simulated with cycle precision in cocotb. We also verify
correctness of the core behaviour by running assembler tests from [riscv-tests](https://github.com/riscv-software-src/riscv-tests/tree/master)
and [riscv-arch-tests](https://github.com/riscv-non-isa/riscv-arch-test).

## Documentation
These three verification steps are automatically run by CI on every commit delivered to the `master` branch. Running
the checks in CI allow us to collect historical data, which are available in the form of the graphs
on a dedicated [benchmark subpage](https://kuznia-rdzeni.github.io/coreblocks/dev/benchmark/).

### Requirements
In CI we use pre-built docker containers, which are publicly available on our [github page](https://github.com/orgs/kuznia-rdzeni/packages).
In the following subsections we provide the instructions on how to manually run verification steps using these containers.
They can be recreated using standard docker build commands:

In order to perform synthesis you will need to install following tools:
* [yosys](https://github.com/YosysHQ/yosys)
* [prjtrellis](https://github.com/YosysHQ/prjtrellis)
* [nextpnr-ecp5](https://github.com/YosysHQ/nextpnr.git)

These tools may need manual compilation from git repository, that can take some time.

You can use docker images that have installed all required tools to perform synthesis:
* [vuush/amaranth-synth:ecp5](https://hub.docker.com/r/vuush/amaranth-synth/tags)

To build the `AmaranthSynthECP5.Dockerfile` yourself use following command:
```
docker build --platform linux/amd64 -t "amaranth-synth:ecp5" -f ./docker/AmaranthSynthECP5.Dockerfile .
docker build --platform linux/amd64 -t "amaranth-synth:latest" -f ./docker/AmaranthSynthECP5.Dockerfile .
```

### Usage

Script named `synthesize.py` is used to perform the `Core` synthesis.

Example usage:
```
./scripts/synthesize.py --help
./scripts/synthesize.py --platform ecp5 --verbose
```
## Synthesis

To collect synthesis information we use script named `parse_benchmark_info.py`.
The basic step in verification is to see if it is possible to synthesise the `Core` circuit. This allows us to
control the level of complexity of the core. Although Coreblocks is an educational core, we want it to be practical.
It should have an acceptable maximum frequency and shouldn't use too many resources, so it can be run
on a FPGA. The synthesis step ensures that these requirements are met. In addition, it checks whether the code that is acceptable
for Amaranth is also acceptable for the synthesis tools.

This script parses the output of the synthesis tool and extracts the
following information:
The main properties collected in the synthesis step:
- Max clock frequency
- Number of logic cells used
- Number of carry cells used
- Number of RAM cells used
- Number of DFF cells used

## Benchmarks
The configuration of the docker container is described in the `AmaranthSynthECP5.Dockerfile`, which can be found in
[our repo](https://github.com/orgs/kuznia-rdzeni/packages/container/package/amaranth-synth).

### Manual reproduction

```bash
sudo docker pull ghcr.io/kuznia-rdzeni/amaranth-synth:latest
sudo docker run -it --rm ghcr.io/kuznia-rdzeni/amaranth-synth:latest
git clone --depth=1 https://github.com/kuznia-rdzeni/coreblocks.git
cd coreblocks
apt update
apt install python3.11-venv
python3 -m venv venv
. venv/bin/activate
python3 -m pip install --upgrade pip
pip3 install -r requirements-dev.txt
PYTHONHASHSEED=0 ./scripts/synthesize.py --verbose --config full
./scripts/parse_benchmark_info.py
cat benchmark.json
```

The main point of the above listing is the `synthesize.py` script, which creates an instance of the `Core` object with
the configuration provided by the user and then passes it to Amaranth to generate a Verilog description from that instance.
This description is then processed by Yosys and nextpnr-ecp5 to generate the ECP5 bitstream.

A strength of Coreblocks is its modularity, so we can provide different configurations with little effort. You can choose
a configuration to synthesise using the `--config` argument to the `synthesise.py` script.

### Dependencies

For each commit on `master` branch, CI runs the synthesis and saves the parameters collected by `parse_benchmark_info` script.
In order to perform synthesis we use:
* [yosys](https://github.com/YosysHQ/yosys) - to synthesise the Verilog generated by Amaranth up to gate level;
* [nextpnr-ecp5](https://github.com/YosysHQ/nextpnr.git) - to perform the "Place and Route" step;
* [prjtrellis](https://github.com/YosysHQ/prjtrellis) - provides the description of the ECP5 bitstream format.

Graphs generated from this information are available on a dedicated [subpage](https://kuznia-rdzeni.github.io/coreblocks/dev/benchmark/).
## Benchmarking

The maximum clock frequency determined by synthesis isn't the only measure of performance. In theory, it is always
possible to increase Fmax by increasing latency. To avoid the pitfall of too long latency affecting the core
throughput, we monitor the number of instructions executed per clock cycle (IPC). We simulate the core with cycle
accuracy and run benchmarks written in C inside the simulation. The benchmarks are taken from
[embench](https://github.com/embench/embench-iot/tree/master).


The benchmarking is done in two steps. First, we compile the C programs into binary format. Second, we run the binaries
on the simulated core. To compile the code we use [riscv-gnu-toolchain](https://github.com/riscv/riscv-gnu-toolchain),
with glibc configured for different architectural subsets of RISC-V extensions (you can check the exact configuration in
[riscv-toolchain.Dockerfile](https://github.com/kuznia-rdzeni/coreblocks/blob/master/docker/riscv-toolchain.Dockerfile)).

Once we have the binaries, we can run them in simulation. This is done using [Cocotb](https://github.com/cocotb/cocotb) and
[Verilator](https://github.com/verilator/verilator). We use Amaranth features to generate Verilog code describing Coreblocks instance,
which is passed to Verilator for compilation. Cocotb controls the simulation and execution of the program by stubbing external
interfaces. Pre-compiled Verilator in a compatible version is available in [Verilator.Dockerfile](https://github.com/kuznia-rdzeni/coreblocks/blob/master/docker/Verilator.Dockerfile).


### Benchmarks manual execution
```bash
# ========== STEP 1: Compilation ==========
# Clone coreblocks into host file system
git clone --depth=1 https://github.com/kuznia-rdzeni/coreblocks.git
cd coreblocks
git submodule update --init --recursive
cd ..
sudo docker pull ghcr.io/kuznia-rdzeni/riscv-toolchain:latest
# Run docker with the coreblocks directory mounted into it
sudo docker run -v ./coreblocks:/coreblocks -it --rm ghcr.io/kuznia-rdzeni/riscv-toolchain:latest
cd /coreblocks/test/external/embench
# Compilation will put binaries in the subdirectory of the /coreblocks directory, which is shared with the host
# so that binaries survive after the docker container is closed
make
exit

# ========== STEP 2: Execution ==========
sudo docker pull ghcr.io/kuznia-rdzeni/verilator:latest
# Run docker with the coreblocks directory mounted into it. This directory contains
# benchmark binaries after running the first step.
sudo docker run -v ./coreblocks:/coreblocks -it --rm ghcr.io/kuznia-rdzeni/verilator:latest
apt update
apt install python3.11-venv
python3 -m venv venv
. venv/bin/activate
python3 -m pip install --upgrade pip
cd coreblocks
pip3 install -r requirements-dev.txt
PYTHONHASHSEED=0 ./scripts/gen_verilog.py --verbose --config full
./scripts/run_benchmarks.py
```

## Regression tests

Regression tests should ensure that Coreblocks is compliant with RISC-V specification requirements. Tests include
assembler programs that tests entire RISC-V instruction set. We execute these programs in a similar way to benchmarks.
So, as a first step, we compile the programs to the binary format and then we run them on core simulated by Verilator
and Cocotb.

### Regression tests manual execution
```bash
# ========== STEP 1: Compilation ==========
# Clone coreblocks into host file system
git clone --depth=1 https://github.com/kuznia-rdzeni/coreblocks.git
cd coreblocks
git submodule update --init --recursive
cd ..
sudo docker pull ghcr.io/kuznia-rdzeni/riscv-toolchain:latest
# Run docker with the coreblocks directory mounted into it
sudo docker run -v ./coreblocks:/coreblocks -it --rm ghcr.io/kuznia-rdzeni/riscv-toolchain:latest
cd /coreblocks/test/external/riscv-tests
# Compilation will put binaries in the subdirectory of the /coreblocks directory, which is shared with the host
# so that binaries survive after the docker container is closed
make
exit

# ========== STEP 2: Execution ==========
sudo docker pull ghcr.io/kuznia-rdzeni/verilator:latest
# Run docker with the coreblocks directory mounted into it. This directory contains
# regression test binaries after running the first step.
sudo docker run -v ./coreblocks:/coreblocks -it --rm ghcr.io/kuznia-rdzeni/verilator:latest
apt update
apt install python3.11-venv
python3 -m venv venv
. venv/bin/activate
python3 -m pip install --upgrade pip
cd coreblocks
pip3 install -r requirements-dev.txt
PYTHONHASHSEED=0 ./scripts/gen_verilog.py --verbose --config full
./scripts/run_tests.py -a regression
```

0 comments on commit 723072b

Please sign in to comment.