|
❗
|
This is currently a work in progress. Because the support for the camera module is under development, the images displayed on the LCD are generated by the PRU firmware. |
This is an experimental embedded Linux OS and application for BeagleBone Black SBC [1] to capture video frames from an OV7670 VGA Camera Module [2] via PRUSS and display them on a 4.3" LCD Display [3].
The main purpose of this project is to continue and improve the work done in the context of Bootlin’s excellent Embedded Linux boot time optimization training [4], which I strongly recommend to anyone looking for solutions to optimize the boot time of Linux-based operating systems.
The boot duration is measured using an Arduino Nano 33 BLE board [5], from BeagleBone Black’s system reset until the first video frame is captured from the camera and presented on the LCD.
The Arduino board starts a timer as soon as the reset signal is received from the BeagleBone board (via a GPIO pin). The timer is stopped when a second GPIO signal is received, which marks the completion of the first video frame capture. The elapsed time is displayed in real time on a Midas 16x2 alphanumeric LCD [6] controlled by Arduino via I2C.
- Current boot time measurements
| Image Format | Resolution | Boot time (s) | Comments |
|---|---|---|---|
CUSTOM |
80 x 60 |
1.230 |
Using real-time test images |
QQVGA |
160 x 120 |
1.265 |
Using real-time test images |
QVGA |
320 x 240 |
1.400 |
Using real-time test images |
CIF |
352 x 288 |
1.455 |
Using real-time test images |
|
ℹ️
|
|
|
ℹ️
|
In the diagram below, the blocks with rounded corners (uncolored) are hardware components, while those with square corners (colored) are software components. The blue colored blocks represent the custom software components developed in the context of this project. |
- Breadboard prototype
The Programmable Real-Time Unit and Industrial Communication Subsystem (PRU-ICSS a.k.a. PRUSS) is present on various TI SoCs, including AM335x which BeagleBone Black relies on.
A PRUSS consists of dual 32-bit RISC cores (Programmable Real-Time Units, or PRUs), shared RAM, data and instruction RAMs, some internal peripheral modules to facilitate industrial communication and an interrupt controller.
The programmable nature of the PRUs provide flexibility to implement custom peripheral interfaces, fast real-time responses or specialized data handling.
|
ℹ️
|
For more details, please refer to BeagleBone Black’s product page [1], the AM335x Technical Reference Manual [7] or elinux.org [8]. |
- PRU 0
-
Used to operate in one of the following modes:
-
Test mode: generates test data for debugging and testing purposes
-
Acquire mode: read image data from the camera module
-
In both modes the data is transfered to PRU 1 via the three scratch pad banks.
- PRU 1
-
Responsible for the communication with the ARM host via the rpmsg infrastructure. Accepts several commands to control the data acquisition process and provides messages with different types of content: information, logs, image data.
|
ℹ️
|
The PRUs firmware source code location is: component/rootfs/br2-external/package/prufw
|
- PRU pins for capturing camera frames via
D0-D7,PCLK,HREFandVSYNC -
There are 16 input pins (and 16 output) pins per PRU core, but not all of these are accessible on the BeagleBone Black. Additionally, some of them are already in use by the LCD cape, hence we can only access less than half of the inputs, as indicated in the following table:
| PRU# | R31 bit | BB Header | BB Pin Name | ZCZ BallName | Pinmux Mode | Cam Pin Name | Comments |
|---|---|---|---|---|---|---|---|
0 |
0 |
P9_31 |
SPI1_SCLK |
mcasp0_aclkx |
Mode_6 |
D0 |
|
0 |
1 |
P9_29 |
SPI1_D0 |
mcasp0_fsx |
Mode_6 |
D1 |
|
0 |
2 |
P9_30 |
SPI1_D1 |
mcasp0_axr0 |
Mode_6 |
D2 |
|
0 |
3 |
P9_28 |
SPI1_CS0 |
mcasp0_ahclkr |
Mode_6 |
See Conflict |
|
0 |
4 |
P9_42 |
GPIO3_18 |
mcasp0_aclkr |
Mode_6 |
D4 |
See Note1 |
0 |
5 |
P9_27 |
GPIO3_19 |
mcasp0_fsr |
Mode_6 |
D5 |
|
0 |
6 |
P9_41 |
GPIO3_20 |
mcasp0_axr1 |
Mode_6 |
D6 |
See Note2 |
0 |
7 |
P9_25 |
GPIO3_21 |
mcasp0_ahclkx |
Mode_6 |
D7 |
|
0 |
14 |
P8_16 |
GPIO1_14 |
gpmc_ad14 |
Mode_6 |
D3 |
|
0 |
15 |
P8_15 |
GPIO1_15 |
gpmc_ad15 |
Mode_6 |
PCLK |
|
0 |
16 |
P9_24 |
UART1_TXD |
uart1_txd |
Mode_6 |
HREF |
|
1 |
12 |
P8_21 |
GPIO1_30 |
gpmc_csn1 |
Mode_6 |
VSYNC |
|
1 |
13 |
P8_20 |
GPIO1_31 |
gpmc_csn2 |
Mode_6 |
See Note3 |
|
1 |
16 |
P9_26 |
UART1_RXD |
uart1_rxd |
Mode_6 |
Not used |
|
ℹ️
|
|
- BeagleBone pins for controlling camera via
XCLK,SIO_CandSIO_D
| BB Header | BB Pin Name | ZCZ BallName | Pinmux Mode | Pinmux Function | Cam Pin Name |
|---|---|---|---|---|---|
P8_07 |
TIMER4 |
gpmc_advn_ale |
Mode_2 |
timer4 |
XCLK |
P9_19 |
I2C2_SCL |
uart1_rtsn |
Mode_3 |
I2C2_SCL |
SIO_C |
P9_20 |
I2C2_SDA |
uart1_ctsn |
Mode_3 |
I2C2_SDA |
SIO_D |
- VGA Frame Timing
The mainline Linux kernel (currently v5.11), with additional PRU related patches adding support for:
-
Triggering an interrupt by signaling a specific PRU system event
-
The optional rpmsg stack using the virtio-ring based communication transport between MPU and a PRU core
|
ℹ️
|
The kernel patches location is: component/linux/patches
|
This is the Linux kernel module responsible for providing a communication interface between user space applications and PRU cores firmware.
For the moment the driver exposes just a rpmsg channel in the form of a raw character device that can be used directly by applications to write/read data to/from PRU cores. Later it might provide additional APIs (e.g. v4l2) to facilitate integration with 3rd-party applications (e.g. ffmpeg, mpv).
|
ℹ️
|
The source code location is: component/rpmsgcam-drv
|
This is the user-space application responsible for:
-
Reading messages from the rpmsg character device
-
Filtering, validating and assembling image frames
-
Displaying the images on the 4.3" LCD via Linux Frame Buffer
-
Signaling the receiving of the first frame via GPIO
|
ℹ️
|
The source code location is: component/rpmsgcam-app
|
Please follow the instructions below to setup your build environment and generate the project binaries: rootfs/initramfs, Linux kernel, U-Boot.
The project building process has been tested on an Ubuntu 20.04 chroot environment,
using schroot, but it should work on any recent Debian based distribution.
Please run the commands below to install all the packages the build environment relies on:
# Required for generating uImage compatible binaries
$ sudo apt install u-boot-tools
# Required for creating/flashing SD card images (dialog, mkdosfs, mcopy)
$ sudo apt install dialog dosfstools mtools
# Required for building the x86 TI's PRU Code Generation Tools (CGT)
$ sudo apt install libc6-i386 lib32stdc++6 lib32z1
# Possibly required for building the Linux kernel
$ sudo apt install kmod libgmp-dev libmpfr-dev libmpc-dev libssl-dev lzop
# Common (usually pre-installed) utilities
$ sudo apt install cpio gawk gettext git openssh-client patch perl python rsync tar unzip wget
# Other (indirect) dependencies
$ sudo apt install bc bison flex genisoimage gperf help2man libncurses-dev libtool-bin texinfo|
❗
|
For other distros (e.g. RPM based), the commands above must be adapted according to the specific package manager and actual package names. |
Let’s assume the project location throughout the document will be ${HOME}/beaglecam.
The simplest approach to get the sources is to clone the upstream repository:
$ cd ${HOME}
$ git clone https://github.com/cristicc/beaglecam.gitAlternatively, you may directly download the source archive:
$ wget https://github.com/cristicc/beaglecam/archive/refs/heads/main.zip
$ unzip main.zip
$ mv beaglecam-main ${HOME}/beaglecam
$ rm main.zip|
💡
|
If $ curl -O https://github.com/cristicc/beaglecam/archive/refs/heads/main.zip |
The project uses a make infrastructure derived from Buildroot and is able
to build most of the components (e.g. Linux kernel, U-Boot) directly.
For building more complex components like toolchain and rootfs, the build platform is using Buildroot internally, but the whole process is automatic (e.g. downloading/configuring/building external dependencies, including buildroot) and no manual operations are required.
$ cd ${HOME}/beaglecam
$ make help
Options:
V=0|1 0 => quiet build (default), 1 => verbose build
O=DIR Create all output artifacts in DIR.
Main targets:
prepare Create build output directories and Makefile wrapper.
all Build project.
clean Delete all files created by build.
distclean Delete all non-source files (including downloads).
reconfigure Rebuild all project components from the configure step.
rebuild Rebuild all project components.
[...]The default build configuration options are stored in prj.config. It is
recommended to keep them unchanged for the first build, to be able to validate
the build environment.
Later you may want to adjust some of the following settings:
PRJ_LINUX_KERNEL_VERSION = a.b.c
PRJ_UBOOT_VERSION = yyyy.mm
PRJ_BUILDROOT_VERSION = yyyy.mm[.bb]Currently the project allows choosing between two build profiles: prod and
dev. The former is implicitly used if the PRJ_PROFILE variable is not set
by the user via the command line or the environment.
By default, the build artifacts will be stored in ${HOME}/beaglecam/output, but this can
be changed via the O=DIR option, for an out-of-tree build.
Now run the following command to initialize the chosen output directory for the
dev profile we are going to use. Additionally we also provide a custom
location for the downloaded source archives to be able to share it between the
two profiles, otherwise every profile will use its own download folder and, as
a consequence, the source packages will be downloaded twice. Also note we call
the prepare target manually to make sure make will not trigger all which
would start the build process. However this is not mandatory since prepare is
implicitly invoked before building any project component.
$ make O=${HOME}/beaglecam/output/dev PRJ_PROFILE=dev DOWNLOAD_DIR=${HOME}/beaglecam/output/dev/../downloads/ prepare
GEN ${HOME}/beaglecam/output/dev/Makefile
$ ls -1a ${HOME}/beaglecam/output/dev
binaries
build
host
Makefile
.stamp_preparedThe binaries folder will contain final images (e.g. kernel, u-boot, rootfs),
the build folder is used for temporary build artifacts and the host folder
will contain the binaries for the host components (e.g. genimage tool).
.stamp_prepared is a timestamp file used internally by the build platform to
avoid redoing the preparation step once completed.
|
💡
|
There is also a Makefile wrapper generated in the custom output folder having
the purpose of simplifying the make usage for out-of-tree builds, i.e. simply
cd to the custom output directory and run make without passing any of the
initial arguments.
|
To build all the project components, just issue the make command in the project
root directory, assuming you are not using the out-of-tree option, otherwise run
the command in the custom output directory.
$ cd ${HOME}/beaglecam/output/dev
$ make
[...]
=== toolchain Installing to binaries directory
[...]
=== linux 5.11.11 Installing to binaries directory
[...]
=== rootfs Installing to binaries directory
[...]
=== uboot 2021.04 Installing to binaries directory
[...]
=== prj 0.1 Rebuilding kernel with initramfs
[...]
=== prj 0.1 Installing to binaries directory
=== prj 0.1 Stripping binaries
=== prj 0.1 Generating bootable SD card imageThe generated images are stored in the binaries folder:
$ ls -1 binaries/
am335x-boneblack-pru.dtb
boot.vfat
MLO
rootfs.cpio
sd-card.img
u-boot.img
uEnv-falcon.txt
uEnv.txt
uImage
zImage|
❗
|
uImage is the kernel image to be used for falcon boot.
For regular boot, the zImage format will be used instead.
|
Insert the micro SD card in a USB card reader attached to the host system and run the following command, assuming the current working directory is still the project output directory:
${HOME}/beaglecam/tools/prepare-sd-card.sh binaries/sd-card.imgYou should see a dialog box displaying the list of all removable USB drives currently accessible from the host system:
Select the correct drive and press OK to start flashing the device using
the storage disk image file (sd-card.img) generated by the build process:
Please wait while writing 'binaries/sd-card.img' to '/dev/sda'.. 50331648 bytes (50 MB, 48 MiB) copied, 2 s, 24.5 MB/s 12+1 records in 12+1 records out 53477376 bytes (53 MB, 51 MiB) copied, 2.21715 s, 24.1 MB/s Done.
Insert the uSD card into BeagleBone SBC and connect the board to the host system using a compatible USB-to-TTL Serial Cable.
Assuming the serial adapter on the host is accessible via /dev/ttyUSB0, you
may use the screen utility to monitor the serial console:
$ screen /dev/ttyUSB0 115200
U-Boot SPL 2021.01 (May 24 2021 - 19:26:29 +0000)
Trying to boot from MMC1
[...]
U-Boot 2021.01 (May 24 2021 - 19:26:29 +0000)
CPU : AM335X-GP rev 2.1
Model: TI AM335x BeagleBone Black
DRAM: 512 MiB
[...]
Starting kernel ...
[ 0.000000] Booting Linux on physical CPU 0x0
[ 0.000000] Linux version 5.11.11 (cristi@ubuntuws) (arm-linux-gcc.br_real (Buildroot 2020.08-14-ge5a2a90) 9.3.0, GNU ld (GNU Binutils) 2.33.1) #7 SMP Sat May 29 21:05:26 UTC 2021
[ 0.000000] CPU: ARMv7 Processor [413fc082] revision 2 (ARMv7), cr=10c5387d
[...]
Starting syslogd: OK
Starting klogd: OK
Running sysctl: OK
Starting dropbear sshd: OK
Starting network: OK
beaglecam login: root
╔══╗ ╔╗ ╔═══╗ ╔═══╦═══╗
║╔╗║ ║║ ║╔═╗║ ║╔═╗║╔═╗║
║╚╝╚╦══╦══╦══╣║╔══╣║ ╚╬══╦╗╔╣║ ║║╚══╗
║╔═╗║║═╣╔╗║╔╗║║║║═╣║ ╔╣╔╗║╚╝║║ ║╠══╗║
║╚═╝║║═╣╔╗║╚╝║╚╣║═╣╚═╝║╔╗║║║║╚═╝║╚═╝║
╚═══╩══╩╝╚╩═╗╠═╩══╩═══╩╝╚╩╩╩╩═══╩═══╝
╔═╝║ Version 0.1 (dev)
╚══╝
root@beaglecam:~#|
💡
|
To make sure BeagleBone is booting from the uSD card, stop at the U-Boot prompt (keep pressing the SPACE key while resetting the board) and run the following commands to erase the partition table of the on-board (e)MMC storage: => mmc list OMAP SD/MMC: 0 (SD) OMAP SD/MMC: 1 => mmc dev 1 switch to partitions #0, OK mmc1(part 0) is current device => mmc erase 0 0x400 MMC erase: dev # 1, block # 0, count 1024 ... 1024 blocks erased: OK |
When building the project via dev profile, the generated OS image provides
networking capabilities and a SSH service listening on the standard port 22.
Login with root/root using any of the connectivity methods described below.
- Ethernet via the RJ45 port
-
Connect BeagleBone board to host PC using an UTP cable. Set the host IP address to
10.0.0.1or anything else in the10.0.0.255sub-network, except10.0.0.100which is used by BeagleBone. - Ethernet via the mini USB port
-
BeagleCamOs is configured to support ethernet over USB link using CDC EEM. After connecting the device to the host system via USB, a new Ethernet network should be detected and listed as
Linux Foundation EEM Gadget. Manually set an IP address in the10.0.1.255sub-network, except10.0.1.100which is already set on BeagleBone side.
|
💡
|
It is also possible to run remote commands without entering the login password,
via the $ ls -l ${HOME}/beaglecam/output/dev/build/rootfs/target/root/.ssh/
authorized_keys beaglecam-id_ecdsaThe ${HOME}/beaglecam/tools/ssh-cmd.sh -o ${HOME}/beaglecam/output/dev cat /etc/os-release
Testing SSH access
Linux beaglecam 5.11.11 #12 SMP Tue Jun 1 15:35:33 UTC 2021 armv7l GNU/Linux
Executing remote cmd: cat /etc/os-release
NAME="BeagleCam OS"
VERSION="0.1 (dev)"
ID=beaglecamos
VERSION_ID=0.1"
PRETTY_NAME="BeagleCam Development OS" |
Perform the following operations in a BeagleBone remote terminal:
- Load PRUSS related kernel modules and mount debugfs
root@beaglecam:~# modprobe -a virtio_rpmsg_bus pru_rproc
[ 9661.513324] remoteproc remoteproc0: 4a334000.pru is available
[ 9661.519675] remoteproc remoteproc1: 4a338000.pru is available
root@beaglecam:~# mount -t debugfs debugfs /sys/kernel/debug- Display PRU1 initial state and registers
root@beaglecam:~# cat /sys/class/remoteproc/remoteproc1/state
offline
root@beaglecam:~# cat /sys/kernel/debug/remoteproc/remoteproc1/regs
============== Control Registers ==============
CTRL := 0x00000001
STS (PC) := 0x00000000 (0x00000000)
[...]- Start PRU1
root@beaglecam:~# echo start > /sys/class/remoteproc/remoteproc1/state
[12385.220140] remoteproc remoteproc1: powering up 4a338000.pru
[12385.233104] remoteproc remoteproc1: Booting fw image am335x-pru1-fw, size 75688
[12385.300276] remoteproc1#vdev0buffer: registered virtio0 (type 7)
[12385.306469] remoteproc remoteproc1: remote processor 4a338000.pru is now up- Display PRU1 updated state and registers
root@beaglecam:~# cat /sys/class/remoteproc/remoteproc1/state
running
root@beaglecam:~# cat /sys/kernel/debug/remoteproc/remoteproc1/regs
============== Control Registers ==============
CTRL := 0x00008003
STS (PC) := 0x0000004c (0x00000130)
[...]|
💡
|
Repeat the steps above for PRU0 by replacing remoteproc1 with remoteproc0.
|
The rpmsgcam-app utility can be used to trigger test data generation on PRU0,
which is passed to PRU1 in 32 byte chunks and eventually read by the application
via the RPMsg bus in frame sections up to 496 bytes in size.
The resolution of the test images is configurable via the program arguments:
root@beaglecam:~# rpmsgcam-app -h
Usage: rpmsgcam-app [-l LOG_LEVEL] [-x CAM_XRES -y CAM_YRES] [-m MAX_FRAMES]
[-c CAM_DEV] [-f FB_DEV] [-r RPMSG_DEV] [-s DUMP_FILE]
[-t [-p PCLK_MHZ]] [-h]
Options:
-l LOG_LEVEL Console log level no (0 FATAL, 1 ERROR, 2 WARN, 3 INFO, 4 DEBUG, 5 TRACE)
-x CAM_XRES Camera X resolution (default 160)
-y CAM_YRES Camera Y resolution (default 120)
-m MAX_FRAMES Exit app after receiving the indicated no. of frames
-c CAM_DEV Camera I2C device path (default /dev/i2c-1)
-f FB_DEV LCD display Frame Buffer device path (default /dev/fb0)
-r RPMSG_DEV RPMsg device path (default /dev/rpmsgcam31)
-g GPIOCHIP_DEV GPIO chip device path (default /dev/gpiochip3)
-o GPIOLINE_OFF GPIO line offset index relative to GPIO chip device (default 31).
The line is used to signal the receiving of the first frame
-s DUMP_FILE File path to save the raw content of the first frame
-t Enable test mode to let PRU0 generate RGB565 images
-p PCLK_MHZ Pixel clock frequency (MHz) for the generated images (default 1)Run the command bellow to generate 320x240 image frames in RGB565 format and
display them on the LCD available via /dev/fb0 frame buffer.
|
💡
|
Use the -m parameter to automatically stop the application after receiving
the indicated no. of frames.
|
root@beaglecam:~# rpmsgcam-app -l 3 -x 320 -y 240 -f /dev/fb0 -r /dev/rpmsgcam31 -m 1 -t -p 2
1970-01-01 04:22:11.473 INFO main.c:496: Starting rpmsgcam app
1970-01-01 04:22:11.473 INFO main.c:513: Initializing LCD frame buffer
1970-01-01 04:22:11.474 INFO fb.c:43: FB screen info: 480x272, 16bpp, xoff=0, yoff=0
1970-01-01 04:22:11.475 INFO main.c:522: Initializing PRUs for 320x240 frame acquisition
1970-01-01 04:22:11.476 INFO PRU:1: Capture configured
1970-01-01 04:22:11.477 INFO main.c:534: Initializing GPIO output line: 31
1970-01-01 04:22:11.497 INFO gpio-util.c:62: Configured GPIO line 31 as output (line name: P9_13 [uart4_txd])
1970-01-01 04:22:11.499 INFO main.c:207: Starting frames acquisition thread
1970-01-01 04:22:11.500 INFO PRU:1: Capture initiated
1970-01-01 04:22:11.501 INFO main.c:306: Starting FB display thread
1970-01-01 04:22:11.679 INFO main.c:241: Received frame: seq=0
1970-01-01 04:22:11.684 INFO main.c:339: Signaled GPIO line: 31
1970-01-01 04:22:11.685 INFO main.c:350: Reached max allowed no. of frames: 1
1970-01-01 04:22:11.686 INFO main.c:287: Stopping FB display thread
1970-01-01 04:22:11.686 INFO main.c:291: Frame display stats: fps=5.4, cnt=1
1970-01-01 04:22:11.699 INFO main.c:573: Stopping rpmsgcam app
1970-01-01 04:22:11.700 INFO main.c:187: Stopping frames acquisition thread
1970-01-01 04:22:11.858 INFO PRU:1: Capture stopped
1970-01-01 04:22:11.859 INFO main.c:191: Frame acquire stats: total=1, dropped=0, discarded=0, rpmsgerr=0The main PRU firmware sources are component/rootfs/br2-external/package/prufw/src/main-pru{0,1}.c
while the related kernel module is component/rpmsgcam-drv/src/rpmsg_cam.c. The corresponding
binaries can be quickly rebuild, redeployed and tested by using the commands below:
# Rebuild firmware for PRU0 & PRU1
$ make rootfs-stg-prufw-rebuild
# Rebuild out-of-tree kernel driver
$ make rpmsgcam-drv-rebuild
# Regenerate rootfs image
$ make rootfs-rebuild
# Deploy images on BeagleBone (drop 'kernel' if config & DTS have not changed)
$ tools/ssh-cmd.sh -t 10.0.1.100 -u rootfs,kernel
# After device rebooted, load the required modules and start the PRU cores
$ tools/ssh-cmd.sh -t 10.0.1.100 'prefix=/sys/class/remoteproc/remoteproc; modprobe -a virtio_rpmsg_bus pru_rproc rpmsg_cam; echo start >${prefix}0/state; echo start >${prefix}1/state;'
# Verify PRU firmware version
$ tools/ssh-cmd.sh -t 10.0.1.100 'printf "%b" "\xbe\xca\x00" >/dev/rpmsgcam31'
$ root@beaglecam:~# cat /dev/rpmsgcam31
# Trigger data capture by sending the START cmd, followed by STOP
$ tools/ssh-cmd.sh -t 10.0.1.100 'printf "%b" "\xbe\xca\x03" >/dev/rpmsgcam31; sleep 1; printf "%b" "\xbe\xca\x04" >/dev/rpmsgcam31'
# View captured data on device terminal
root@beaglecam:~# cat /dev/rpmsgcam31 | hexdump -CTo speed up the development process, it is possible to test the PRU firmware and/or the kernel driver without rebuilding the rootfs and rebooting the device:
# Transfer and reload the kernel module
$ scp ${HOME}/beaglecam/output/dev/build/rpmsgcam-drv/rpmsg_cam.ko [email protected]:/lib/modules/5.*/extra/
$ tools/ssh-cmd.sh -t 10.0.1.100 'modprobe -r rpmsg_cam; modprobe -a virtio_rpmsg_bus pru_rproc rpmsg_cam'
# Transfer the PRU firmware files to device and restart PRU cores
$ scp ${HOME}/beaglecam/output/dev/build/rootfs/target/lib/firmware/* [email protected]:/lib/firmware/
$ tools/ssh-cmd.sh -t 10.0.1.100 'prefix=/sys/class/remoteproc/remoteproc; echo stop >${prefix}0/state; echo start >${prefix}0/state; echo stop >${prefix}1/state; echo start >${prefix}1/state'|
|
The content of the files copied via scp command above is not written to media
storage, therefore it is necessary to run all the steps again whenever the device
is rebooted.
|
This operation mode allows U-Boot MLO (SPL) to skip loading u-boot.img and
instead load and start the Linux kernel directly.
To get the best possible performance, build the project using the prod profile:
$ make O=${HOME}/beaglecam/output/prod PRJ_PROFILE=prod DOWNLOAD_DIR=${HOME}/beaglecam/output/prod/../downloads/ prepare
$ cd ${HOME}/beaglecam/output/prod
$ makeOnce the bootable SD card image has been generated, follow Prepare a bootable uSD card section and boot the device.
Note the Falcon mode is not yet active since it currently requires some manual steps. Hence, restart the device and stop at the U-Boot prompt by pressing a key within the two seconds default autoboot delay:
U-Boot 2021.01 (Jul 23 2021 - 13:41:15 +0000) [...] Hit any key to stop autoboot: 0 =>
Enter the following U-Boot commands:
=> load mmc 0:1 ${loadaddr} uEnv-falcon.txt
427 bytes read in 2 ms (208 KiB/s)
=> env import -t ${loadaddr} ${filesize}
=> run enable_falcon
2726600 bytes read in 178 ms (14.6 MiB/s)
66560 bytes read in 6 ms (10.6 MiB/s)
## Booting kernel from Legacy Image at 82000000 ...
Image Name: Linux-5.11.11
Created: 2021-05-24 20:20:53 UTC
Image Type: ARM Linux Kernel Image (uncompressed)
Data Size: 2726536 Bytes = 2.6 MiB
Load Address: 80008000
Entry Point: 80008000
Verifying Checksum ... OK
## Flattened Device Tree blob at 88000000
Booting using the fdt blob at 0x88000000
Loading Kernel Image
Loading Device Tree to 8ffec000, end 8ffff3ff ... OK
subcommand not supported
subcommand not supported
Loading Device Tree to 8ffd5000, end 8ffeb3ff ... OK
Argument image is now in RAM: 0x8ffd5000
WARN: FDT size > CMD_SPL_WRITE_SIZE
69632 bytes written in 61 ms (1.1 MiB/s)
Saving Environment to FAT... OK
U-Boot Falcon mode should be enabled now. Restart the board, either from the RESET button or from U-Boot console.
=> reset resetting ...
Note the first image frame should be displayed pretty fast, in ~1.2 seconds when using QQVGA format (160 x 120 pixels). As a matter of fact, the whole process from starting U-Boot up to having the first frame presented takes only ~0.8 seconds, as measured by grabserial utility:
$ grabserial -d /dev/ttyUSB0 -m "U-Boot SPL" -t -e 15 -q "Signaled GPIO line" [0.000000 0.000000] [0.000308 0.000307] U-Boot SPL 2021.01 (Jul 27 2021 - 10:22:42 +0000) [...] [0.712560 0.640093] Starting PRUs [0.715645 0.003085] Probing camera drv [0.728098 0.012453] Starting camera app [0.729740 0.001642] 1970-01-01 00:00:00.230 INFO main.c:478: Starting rpmsgcam app [...] [0.754141 0.009126] 1970-01-01 00:00:00.230 INFO main.c:504: Initializing PRUs for 160x120 frame acquisition [...] [0.798179 0.008390] 1970-01-01 00:00:00.232 INFO PRU:1: Capture initiated [...] [0.819581 0.007253] 1970-01-01 00:00:00.287 INFO main.c:333: Signaled GPIO line [...]
|
|
For some reason it takes ~0.4 seconds to perform the hardware reset and run the ROM code that loads and starts U-Boot SPL. I’m not sure this is a hardware fault or just normal and expected behavior. |
Although this is still work in progress, there are a number of issues that should be addressed sooner or later:
-
Software shut down is not working properly, the board remains powered on as indicated by the blue LED. Please note this functionality is currently only available for
devbuild profile. -
U-Boot crashes when trying to enable Falcon mode with
zImagekernel format. The workaround is to use the legacyuImageformat instead.
-
Add v4l2 support to the Linux kernel camera driver, to facilitate integration with 3rd-party applications (e.g. ffmpeg, mpv)
-
Add support for additional boards (e.g. RaspberryPi, BeagleV)
-
[1] BeagleBone Black product page: https://beagleboard.org/black
-
[2] OV7670 Camera Module: https://www.optimusdigital.ro/en/optical-sensors/624-modul-camera-ov7670.html
-
[3] BeagleBone 4.3" LCD Display Cape: https://www.element14.com/community/docs/DOC-81966
-
[4] Bootlin’s Embedded Linux boot time optimization training: https://bootlin.com/training/boot-time/
-
[5] Arduino Nano 33 BLE: https://store.arduino.cc/arduino-nano-33-ble
-
[6] Midas 16x2 I2C Alphanumeric LCD: https://uk.farnell.com/midas/mccog21605b6w-fptlwi/lcd-cog-16x2-i2c-fstn-blk-on-white/dp/2218942
-
[7] AM335x and AMIC110 Sitara™ Processors Technical Reference Manual: https://www.ti.com/lit/ug/spruh73q/spruh73q.pdf
-
[8] Ti AM33XX PRUSSv2: https://elinux.org/Ti_AM33XX_PRUSSv2