Figure 1: Basys3
Our project involves implementing a Spiking Neural Network (SNN) on an FPGA for real-time image processing using VHDL. The SNN mimics the spiking behavior of biological neurons, addressing the demand for efficient and low-power neural network solutions in embedded systems. Designed with VHDL to leverage FPGA parallelism, our model undergoes a crucial training phase in Python for weight calculation and data normalization, enhancing its ability to discern visual features.
The FPGA acts as a hardware accelerator for image processing, and the SNN is optimized for color detection, specifically distinguishing between crosswalk and road areas. Real-time observation and behavioral analysis offer insights into system accuracy. Performance metrics, including accuracy and latency, evaluate the efficiency of the SNN on FPGA.
Figure 2: Source - Medium (https://medium.com/@tapwi93/first-steps-in-spiking-neural-networks-da3c82f538ad)
Spiking-Neural-Network-on-FPGA/
|-- SNN-RGB/
|-- Sources/
| |-- images/
| |-- Octave/
|-- Py Scripts/
|-- Schematic/
|-- README.md
The system architecture includes an input layer for processing RGB values, a hidden layer with seven neurons, and an output layer designed for white color detection. Octave was initially employed for neural network training, generating weights used in VHDL design through Vivado. Python facilitated weight normalization for consistency. Real-world image integration utilized the PPM format, translating RGB values seamlessly into the VHDL system's 8-bit representation. Vivado implementation involved a structured input delivery, producing an output after 64 cycles and resulting in a block-like structure in the detected image. Rigorous testing, using a test image, validated the system's performance, assessing its accuracy in detecting white color and confirming its robustness in real-world scenarios.
PPM, which stands for Portable Pixel Map, is a widely used image file format that stores images in a simple, human-readable, and portable manner. It is a part of the Netpbm graphics toolkit and is supported by various image processing and editing tools.
| Training Image | Labelled Image |
|---|---|
 |
 |
We developed several modules, including control, gen_input, and neuron, integrated into the top module named snn_rgb. The overall architecture, depicted in the diagram, involves three major inputs (R, G, B), seven hidden nodes, and two output nodes. This neural network processes RGB inputs and ensures effective communication between modules. Testing and validation have been conducted to optimize functionality, with a focus on refining the architecture for desired project performance.
| Test Image | Output Image |
|---|---|
 |
 |
By observing the detected output image, the road and similar darken colored one are detected as green and the white crosswalk is detected as red. And some other parts of the image are red due to their similarity with the white color. There is a block like structure due to the cycle issue mentioned earlier.
We need to do some research on how to resolve the issue and find some alternative ways. We just detected the colors only for detection of crosswalk. As an immediate next step, we can use video frames with high-speed detection and use the same technique to alert the drivers in the event of traffic light color changes.