- Concepts Learnt
- QuickStart for automation (WIP)
- Virtualisation
- OpenNI SDK
- ROS
- Jetson Orin Nano
- Python
- Docker (udev linux device manager does not work causing inability to use depth camera)
- Issues
- Author
- ROS Basics: Learn to navigate the ROS environment, including setting up packages, nodes, and topics.
- Linux Command Line: Master essential Linux commands like
ls,cd,grep, andchmodthat are frequently used in ROS workflows. - Depth Camera Setup: Understand how to connect and configure depth cameras in a ROS environment, including troubleshooting common issues.
- OpenNI and OpenCV: Explore the usage of OpenNI for depth sensing and OpenCV for image processing, and how they integrate with ROS.
- Python Scripting in ROS: Learn how to write Python scripts that interact with ROS nodes, process depth data, and handle camera outputs.
- Virtualisation for ROS: Quick development when software or hardware takes time to setup though lack completeness
- Run
roslaunch astra_camera astra.launch - Run
python3 ultrasonic_sensor.py - Run
python3 detect_silo.py - Run
python3 ros2serial_gripper.pyandpython3 ros2serial_gripper.py - Turn on motor and gripper power
- Download a VM of your choice (Oracle VirtualBox is used)
- Download Ubuntu 22.04 LTS Jammy Jellyfish iso image (so its compatible with ROS2 Humble)
- Setup the Ubuntu VM (at least 4GB RAM but 8GB is better, minimum 25GB storage, 4CPU, USB of depth cam need to be added)
- Once in Ubuntu, download ROS2 Humble following here (all commands are typed in terminal can press window key and search for it)
- Open firefox and go to here & scroll to the bottom under
Wrappers for OpenNI SDKto download OpenNI SDK for ROS2 - After download the tar zip file, extract it & open the README.md. Follow the instructions in README.md. (
IMPORTANT: Change everygalacticword in the file tohumble) - Halfway through you might run into an error with a command starting with
colconbecause its not installed. Follow the steps from here to download it. - At the end, you should see a program with black and white image
- Download VM
- Download Ubuntu 20
- Setup VM (Remember to add usb to vm)
- Follow steps from here to setup ROS here
- Follow steps from here to setup OpenNI here
- If using virtualbox vm after setting up vm go to setting usb and add the usb (must be plugged in the same physical port)
- USB controller of VM is set to usb3.0 (has smoother video stream)
- This might need to be rerun everytime for it to detect the camera
roscd astra_camera ./scripts/create_udev_rules sudo udevadm control --reload && sudo udevadm trigger
- Use below command
rosservice call /camera/save_images "{}"- Images are saved in
~/.ros/imageand are only available when the sensor is on. - xdg-open
- Replace with filename
It uses custom made msg so need to follow this:
- Create Coords.msg and CoordsMatrix.msg in
/home/ros/ros_ws/src/ros_astra_camera/msg - Copy contents of both files from
msgfolder here into your local machine - Follow the steps here to build the custom msg link
- Installation not needed since all necessary dependencies are downloaded in
ros_astra_camerapackage - Get into
Scriptsfolder inros_astra_camera
cd ~/ros_ws/src/ros_astra_camera/scripts- Create a python file
touch detect.py- Paste code from detect.py file into your file in VM
gedit detect.py- Save the file and give it execution permission
chmod +x detect.py- Be sure to source these files on all the terminal before running Step 7-8
cd ~/ros_ws
source /opt/ros/noetic/setup.bash
source ~/ros_ws/devel/setup.bash- To not do this for all terminal
gedit ~/.bashrc - Copy below into the very bottom of the file and save
source /opt/ros/noetic/setup.bash source ~/ros_ws/devel/setup.bash
- Run the depth camera
source ./devel/setup.bash
roslaunch astra_camera astra.launch- Open a terminal and run
roscore- Open another terminal and run
rosrun ros_astra_camera detect.py- Using a linux machine (or maybe VM not tested), download Nvidia SDK Manager
- Use Jumper Wire to connect Ground and Force Reset Pin on Jetson Orin Nano
- Connect type C from Jetson to linux machine
- Provide Jetson Power to boot in (safe mode/reset mode?)
- Install with SDK Manager
Purpose is to create an isolated environment for python packages so there is no risk of dependencies conflict with other similar packages
- Install virtual env(venv) package
pip install virtualenv- Create venv environment
python -m venv .venv- Activate env
- Windows
.venv\Scripts\activate
- Linux/Mac
source .venv/bin/activate - Install saved dependencies
pip install -r requirements.txt- To leave env
deactivate- Once env is activate can use pip install as usual to install new dependencies
- To check dependencies use
pip list- After adding new dependencies execute below to save dependencies
pip freeze > requirements.txt- Download docker desktop from the site
- Follow instructions here
- vcxsrv X server is required to run gui through x11 forwarding in ssh, install
- Putty (optional)
Replace anything in <>
- Pull the docker image from dockerhub
docker pull shadowofskull/depth_cam:latest- Run a container from the image
docker run -it -p 22:22 --name ros-x11-ex shadowofskull/depth_cam:latest- To check container id
docker ps- To add extra terminal
docker exec -it <container id> bash- To rerun already created container use
docker start <container id> - Get into the folder Dockerfile exists
- Run
docker build -t <image name> . - Add --no-cache if having issue building
- Q:Color image not showing
- A: Issue lies in ros2 and openni sdk for ros2, use back ros instead
- ROS2 openni seems to not support uvc camera color stream which our depth cam seems to be
- check if can activate uvc like in ros1 or not
- Unable to synchronise Arduino and Python code through serial.
- Lack of real world and virtual world Gazebo testing cause automation to fail
- Remote disconnection issue potentially due to limitation of ssh or internet connection of field.