Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Proof-of-concept Nix environments for Ibex development #2156

Draft
wants to merge 19 commits into
base: master
Choose a base branch
from

Conversation

hcallahan-lowrisc
Copy link
Contributor

@hcallahan-lowrisc hcallahan-lowrisc commented Mar 27, 2024

This PR is just a PoC, and will likely never be merged. It contains some unrelated changes that I didn't bother completely tidying up, and some short-term fixes to make things work. Hopefully it is obvious which parts are unrelated.

Add a Nix flake to provide development environments for Ibex simulation and synthesis.
The goal of this is to try and utilize the principled properties of Nix (the tool, the language, etc) to easily bootstrap reproducible development environments. This should work on any x86_64-linux system where nix is installable (macOS support is also possible), as there is no dependency on system libraries at all.

Precursors

  1. Install Nix (I suggest to use this guide: https://zero-to-nix.com/start/install). If you use another installer, you will need to ensure the flakes and nix-command experimental features are enabled.
  2. Clone/checkout this branch
git clone [email protected]:hcallahan-lowrisc/ibex.git --branch nix_env_init_poc
cd ibex/

N.B. The first time running nix develop may take a little while to fetch and build dependencies. This is normal (no system library deps etc.), and the second time everything will be cached.

Demo 1 (verilator sims)

  1. Enter a development shell
nix develop # This drops you into a subshell with all the deps on your path
# When you are done with the devshell...
exit
  1. Run some verilator sims (My makefile changes are incomplete, so ensure you clean with make reset when switching between sims)
# Run the simple_system under verilator
make run-simple-system-hello
# Run coremark under verilator
make run-coremark
# Run a pmp smoketest
make run-pmp-smoke-test

Demo 2 (synthesis)

  1. Enter a different development shell, which contains the synthesis deps
nix develop .#syn_shell
  1. Run a yosys synthesis using the Nangate45 open cell library
make run_syn_yosys_nangate45

Demo 3 (HDL simulations including bootstrapping internal lowrisc tools)

N.B. this will only evaluate for lowrisc internal environments with appropriate credentials. See the commit message (350969a) for more details

  1. Enter a development shell
nix develop .#eda_shell_lowrisc
  1. Run a simulation using the core_ibex UVM environment
make -C dv/uvm/core_ibex SIMULATOR=xlm ITERATIONS=4 TEST=riscv_rand_instr_test

- Changes in /vendor should be applied using a vendoring patch, or fixed upstream

Signed-off-by: Harry Callahan <[email protected]>
This improves portability across different operating systems

Signed-off-by: Harry Callahan <[email protected]>
There was problems running vcs simulations with this command, as something to do
with launching verdi is not working correctly. It seems we are missing deps for
some executable, but the error messages are really unhelpful on tracking down
exactly what. This seems to work, for now.

Signed-off-by: Harry Callahan <[email protected]>
This ensures any logging we do before the call to 'run_one()' will appear in the
correct order in the file.

Signed-off-by: Harry Callahan <[email protected]>
This is in-line with how we handle other environment variables in the build, and
we set the variable inside the appropriate dictionary when invoking the
command (run_rtl.py in this case).
This matches the dependencies given in the python-requirements.txt file in the
repo root, but uses the pyproject.toml format and the poetry tool to allow nix
to import this information to construct a python environment.

The poetry.lock file allows each python dep to be pinned, and would need to be
updated using the poetry tool to bump any dependency.

e.g.
```
nix shell nixpkgs#poetry
poetry update
```
@hcallahan-lowrisc
Copy link
Contributor Author

@GregAC @rswarbrick you may find this interesting to play around with
@nbdd0121 @HU90m if you have any ideas about where we could go next!

- The default devShell has enough dependencies to run verilator simulations of Ibex
- Some other dependencies are setup but yet unusued.
@hcallahan-lowrisc hcallahan-lowrisc force-pushed the nix_env_init_poc branch 4 times, most recently from 353d23b to d8cad20 Compare March 29, 2024 09:01
This adds a devShell that employees can use to bootstrap access to non-public
EDA tooling. Evaluation will fail without appropriate credentials to fetch
the repository 'lowrisc-nix-private'.

Disclaimer: EXPERIMENTAL

These shells will only be functional in the appropriate restricted environments.

This is an experiment at tracking dependencies on proprietary tooling that is
less out-of-band compared to simply assuming the underlying environment is
pre-populated. For obvious reasons, this will always have much weaker
reproducibility guarantees than freely-available software and open-source deps.
However we can still lean on Nix to make bootstrapping non-public environments
fast, ergonomic and hopefully reproducible within the constricted space.

Using a nix flake input that is a private repository, we can effectively pin a
version of the private dependencies (hash+timestamp etc) without exposing what
they are. As nix is lazily evaluated, these inputs will not attempt to be
fetched unless we evaluate an output which depends on them, and hence they
should happily co-exist with other flake attributes for most consumers.

To avoid the flake.lock in this repository from exposing the transitive deps of
the private input, that flake does not track it's inputs in the standard way.
Hence, impure evaluation mode is required when using these outputs.

e.g.
```
nix develop .#eda_shell_lowrisc
nix develop .#eda_shell_lowrisc --command bash -c \
"make -C dv/uvm/core_ibex SIMULATOR=xlm ITERATIONS=4 TEST=riscv_rand_instr_test"
```

Signed-off-by: Harry Callahan <[email protected]>
Copy link
Member

@HU90m HU90m left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ran this locally and it worked great!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants