Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improving documentation for porting seL4 and making it up-to-date #222

Merged
merged 1 commit into from
Mar 25, 2024
+169 −43
Merged

Improving documentation for porting seL4 and making it up-to-date #222

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
212 changes: 169 additions & 43 deletions projects/sel4/porting.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7,79 +7,205 @@ SPDX-FileCopyrightText: 2020 seL4 Project a Series of LF Projects, LLC.

# Porting seL4 to a new platform

This section covers porting seL4 to a new ARM or RISC-V platform. You can check the list of supported seL4 platforms [here](https://docs.sel4.systems/Hardware/).

## Setup

In order to load the seL4 image (via some means) it is important to have a working bootloader.
We tend to use [U-boot](https://www.denx.de/wiki/U-Boot) as it is open source, reliable and supports many different architectures.
We tend to use [U-Boot](https://www.denx.de/wiki/U-Boot) as it is open source, reliable and supports many different architectures.

There are a variety of ways of loading an seL4 image from U-Boot such as eMMC, TFTP, and USB.

### TFTP setup

For initial setup you can follow the instructions on the UNSW advanced operating systems website regarding [setting up a tftp server](https://www.cse.unsw.edu.au/~cs9242/19/project/linux.shtml).
For setting up a TFTP server to transfer images over the network, you can follow the
[UNSW Advanced Operating Systems course guide](https://www.cse.unsw.edu.au/~cs9242/19/project/linux.shtml).

Once you get U-boot up and running it is a good idea (if the platform is supported) to flash a
Linux image onto the dev board. This way you can find the correct address to load the image into memory.
Once you get U-Boot up and running it is a good idea (if the platform is supported) to flash a
Linux image onto the board. This way you can find the correct address to load the image into memory.
This is also useful for debugging, as you can use the FDT file system to dump addresses from a running system.

Once you have found the correct address to load, you can use the command `bootcmd="tftpboot 0x20000000 sel4_image && go 0x20000000"`
(using 0x20000000 as an example) followed by `saveenv` to save the starting address onto the boards flash memory.
You will also need to set the 'ipaddr' and 'serverip' details can be found in the link above.
Once you have found the correct address to load, declare the boot command (where address 0x20000000 as an example):
```sh
bootcmd="tftpboot 0x20000000 /path/to/image && go 0x20000000"
saveenv
```

## seL4
For TFTP, you will also need to set some U-Boot envrionment variables:
```sh
setenv ipaddr <BOARD ADDRESS>
setenv serverip <SERVER ADDRESS>
saveenv
```

### eMMC/SD card setup

In order to have a successful port of seL4 to your target ARM platform you will need to go through the following steps.
We can load an image with the following command (using `0x20000000` as an example):
```sh
fatload mmc 0 0x20000000 /path/to/image
```

### DTS
You can find more details in the [U-Boot documentation](https://docs.u-boot.org/en/latest/usage/cmd/fatload.html).

Look for DTS and drivers inside the [Linux repository](https://github.com/torvalds/linux), normally they will be supported. If they are supported clone Linux and modify the `update-dts.sh` to add your target. Run the script located in `kernel/tools/dts/update-dts.sh` at the cloned Linux repository. This will create a "platform.dts" file from Linux inside the dts folder in the kernel. You may need to add device nodes depending on what Linux uses and what is required by seL4. For example, in the port of the Rockpro64 a memory node and an extra timer node were needed so these had to be defined manually.
## seL4

In order to have a successful port of seL4 to your target ARM or RISC-V platform you will need to go through the following steps.

### Device Tree

Device Trees are used by ARM and RISC-V platforms to describe the hardware platform.
seL4 makes use of Device Trees at build time, which is why acquiring a Device Tree Source (DTS)
for your platform is one of the first steps.

seL4 relies on the DTS for the location and size of main memory as well as information
about certain devices used by the kernel such as the timer. In debug mode, seL4 also
makes use of the default serial device for debug output.

Typically the Device Tree Source will be included in the
[Linux kernel source](https://github.com/torvalds/linux), from there you can decompile
the device tree blob (DTB) from building the Linux kernel to get the Device Tree that
Linux uses.

You can compile the Linux Device Tree Sources into a final Device Tree Blob with:
```sh
make ARCH=<arm/arm64/riscv> CROSS_COMPILE=<TOOLCHAIN> defconfig
make ARCH=<arm/arm64/riscv> CROSS_COMPILE=<TOOLCHAIN> dtbs
```

The DTBs can be found in `arch/<ARCH>/boots/dts/`.

The following command allows you to decompile the DTB:
```sh
dtc -I dtb -O dts /path/to/dtb -o linux.dts
```

It should be noted that you may need to add device nodes depending on what Linux uses
and what is required by seL4. For example, some platforms do not include a node for
main memory in the Linux Device Tree. This is because Linux can get
this information at run-time from a previous bootloader. This is not the case for seL4
as it consumes the Device Tree at build-time.

In the seL4 source, each platform has an overlay DTS that is responsible for dealing
with seL4-specific device nodes. Below is an example
(from the [ROCKPro64](https://github.com/seL4/seL4/blob/master/src/plat/rockpro64/overlay-rockpro64.dts)):
```
/ {
chosen {
seL4,elfloader-devices =
"serial2",
&{/psci},
&{/timer};
seL4,kernel-devices =
"serial2",
&{/interrupt-controller@fee00000},
&{/timer};
};
};
```

These overlay files give a good idea of what nodes seL4 requires in the DTS other than
main memory.

You will need to create a similar overlay DTS in `src/plat/<platform>/overlay-<platform>.dts`
as part of the platform port.

### Hardware generation script

Depending on your platform and the way the DTS is defined you may need to modify the hardware_gen.py script.
From the above example the Rockpro64 needed a memory node so that this script could define working memory.
Other gotchas could include interrupt cells, interrupt numbers being mapped wrongly, or incorrect addresses.
Also serial and timer drivers will need to be added to the kernel if they don't already exist. You can check this through
the DTS device strings and compare to the hardware.yml file in `kernel/tools/hardware.yml`. If needed add the required drivers in `kernel/drivers/`.
Depending on your platform and the way the DTS is defined you may need to modify the
`seL4/tools/hardware_gen.py` script.

From the above example the ROCKPro64 needed a memory node so that this script could
define working memory. Other gotchas could include interrupt cells, interrupt numbers being
mapped wrongly, or incorrect addresses. Also serial and timer drivers will need to be added
to the kernel if they don't already exist. You can check this through the DTS device strings
and compare to the `seL4/tools/hardware.yml` file.

If needed, add the required drivers in `seL4/src/drivers/` and `seL4/include/drivers/`.

### Kernel

You will need to add `libsel4/sel4_plat_include/<platform>/sel4/plat/api/constants.h` and add sel4_UserTop depending on what you support. Look in the other platform files for examples similar to your platform.
You will need to add `seL4/libsel4/sel4_plat_include/<platform>/sel4/plat/api/constants.h`.
Depending on your platform, this is either empty or requires defining the value for `seL4_UserTop`.
Look in the other platform files for examples similar to your platform.

### CMake-build system
You will need to add a CMake configuration file for the platform at `kernel/src/plat/<platform>/config.cmake`.
It is best to base the contents of this file off of other platforms of the same architecture.

You will need to modify the build system in two places. Firstly in the kernel's platform folder, you define `kernel/src/plat/<platform>/config.cmake`, again you can look at other platforms for examples.
Secondly, you will need to add 'KernelPlatform' to the cmake and elf-loader tools. You can do this by editing
`tools/cmake-tool/helpers/application_settings.cmake` and `tools/elfloader-tool/CMakeLists.txt`.
## Testing your seL4 port

In order to sanity check your port of seL4, it is a good idea to get
[the seL4test project](https://docs.sel4.systems/projects/sel4test/) running and passing.
lsf37 marked this conversation as resolved.
Show resolved Hide resolved

### ELF-loader

You will need to add directories and files `tools/elfloader-tool/src/plat/<platform>/sys_fputc.c` and
`tools/elfloader-tool/include/plat/<platform>/platform.h` which will provide a simple implementation of putchar with a physical address for uart for the elf loader to use.
Before we can run sel4test, we need to have seL4 booting first.

Although we use U-Boot as a bootloader, there are additional steps needed to initialise the
hardware and setup system images before we can boot seL4. This is the purpose of the
[ELF-loader](https://docs.sel4.systems/projects/elfloader/).

## seL4 test
The code for the ELF-loader can be found in the `elfloader-tool` directory at the root of
the [seL4_tools](https://github.com/seL4/seL4_tools) repository.

seL4test needs user level serial and timer drivers to run. Add these drivers and the following files to libplatsupport in
order to get the test suite running. The directory structure and files will look something like the following. Some of these
files may not have anything in them but include them anyway to keep the build system happy.
Depending on your platform, you may have to make changes to the ELF-loader, which is
responsible for loading the kernel and any user-programs. The ELF-loader outputs to the
serial, so needs to be able to access the UART.

Use the other platforms as examples; generally you will have to configure the timer driver by reading the manual, it may have its
own unique programming sequence and can sometimes be a bit tricky.
On RISC-V, outputting to UART is done via a call to the SBI layer so no changes are necessary.

You can find the physical addresses and interrupt numbers for each device in its platform reference manual and copy them into the right places.
On ARM, there are a variety of existing UART drivers in `tools/elfloader-tool/src/drivers/uart/`.
If the default serial is not supported by any of these drivers, you will have to
provide a simple putchar implementation.

`libplatsupport/plat_include/<platform>/`
-`clock.h`
-`gpio.h`
-`i2c.h`
-`mux.h`
-`serial.h`
-`timer.h`
`libplatsupport/src/plat/<platform>/`
-`chardev.c`
-`ltimer.c`
-`serial.c`
-`timer.c`
If the system image that is produced by the ELF-loader build system needs be a particular file
format (e.g binary or EFI) or needs to start at a particular physical address, you can
configure the default platform options in `seL4_tools/cmake-tool/helpers/application_settings.cmake`.

### seL4test

seL4test needs user level serial and timer drivers to run. Add these drivers and the following
files to [libplatsupport](https://github.com/seL4/util_libs/tree/master/libplatsupport)
in order to get the test suite running. The directory structure and
files will look something like the following. Some of these files may not have anything in
them but include them anyway to keep the build system happy.

Use the other platforms as examples; generally you will have to configure the timer driver by
reading the manual, it may have its own unique programming sequence and can sometimes be a bit tricky.

If the technical reference manual does not contain enough detail, you may be able to gain more
information by looking at the respective Linux driver for the device. This can be done by searching
for the value of the 'compatible' field that is on each device node in a DTS in the Linux source code.

You can find the physical addresses and interrupt numbers for each device in its platform reference
manual and copy them into the right places.

`util_libs/libplatsupport/plat_include/<platform>/`
* `clock.h`
* `gpio.h`
* `i2c.h`
* `mux.h`
* `serial.h`
* `timer.h`

`util_libs/libplatsupport/src/plat/<platform>/`
* `chardev.c`
* `ltimer.c`
* `serial.c`
* `timer.c`

### sel4test configurations

The default configuration of seL4 in sel4test is non-MCS, single-core, with no hypervisor mode.

After confirming that the default configuration successfully runs (in both debug and release mode),
it is a good idea to test the more complex configurations with your port as well.

### Gotchas

Incorrect address alignment and incorrect memory regions can cause instruction faults. One way to debug this, is
to shorten memory regions in DTS memory nodes to check you are touching the correct area.

Once you have succeeded with your port and have seL4 test passing in release mode, you can add an entry to the CHANGES file describing the platform you added and submit a pull request on the seL4 github.
## Contributing

Once you have a working port that passes seL4test, see the guides for [contributing in general]
and [contributing kernel changes](https://docs.sel4.systems/processes/contributing.html).