Image recognition and tracking based on TensorFlow-Lite-Object-Detection-on-Android-and-Raspberry-Pi and control a gimbal to track the recognized object at the center of image. Gimbal based on PCA9685. Also send message on MAVLink.
To run this code: python3 TFLite_detection_webcam.py
Required hardware:
- Raspberry Pi 4 Computer Model B 8GB Single Board Computer Suitable for Building Mini PC/Smart Robot/Game Console/Workstation/Media Center/Etc.
- Coral USB Accelerator
- Raspberry Pi Camera Module V2-8 Megapixel,1080p (RPI-CAM-V2)
- Pixhawk PX4 PIX 2.4.8 32 Bit Flight Controller Autopilot with 4G SD Safety Switch Buzzer PPM I2C for RC Quadcopter Ardupilot
- Arducam Upgraded Camera Pan Tilt Platform Compatible with Raspberry Pi Camera Module 3/V1/V2, Nvidia Jetson Nano/Xavier NX
Required software packages:
- Install Rasbian Headless (32-bit Bullseye) on RasPi4B
- Install TensorFlow-Lite-Object-Detection from Google Coral
- Install ArduCam
- Install MavLink To install these packages, simply run: $ sh get_pi_requirements.sh in the package.
References:
- https://www.tomshardware.com/reviews/raspberry-pi-headless-setup-how-to,6028.html
- https://github.com/EdjeElectronics/TensorFlow-Lite-Object-Detection-on-Android-and-Raspberry-Pi/blob/master/deploy_guides/Raspberry_Pi_Guide.md
- https://github.com/ArduCAM/PCA9685
- https://ardupilot.org/dev/docs/raspberry-pi-via-mavlink.html
Raspberry Pi4 HW Configuraton in Raspi-config:
sudo raspi-config
- select: 3 Interface Options Configure connections to peripherals.
- select: I1 Legacy Camera Enable/disable legacy camera support.
select: Yes - select: I6 Serial Port Enable/disable shell messages on serial connection. select: No to Would you like a login shell to be accessible over serial? select: Yes to Would you like the serial port hardware to be enabled?
- select: 3 Interface Options Configure connections to peripherals. select: I5 I2C Enable/disable automatic loading of I2C kernel module. select: Yes
- select: Finish but don't reboot.
-
sudo vi /boot/config.txt append dtoverlay=disable-bt enable_uart=1
-
sudo vi /etc/pip.conf append break-system-packages = true
-
reboot
Wiring between Raspberr Pi 4 and Pixhawk 4: https://docs.px4.io/main/en/companion_computer/pixhawk_rpi.html
Download this code and run the shell script: git clone https://github.com/winston1408/tflite1-mavlink.git cd tflite1-mavlink/ sh get_pi_requirements.sh