OAK-FFC-6P Running a Python code on the camera

bednar

Hello, I would like to run a custom VIO code onboard the camera. Is there a way how to run the algorithm there?

I have only found some examples specifying using the DepthAI Python with rvc3_support branch but it has to be installed on the host computer/robot, not the camera right?

Is there any sample Python code that could be run onboard?

erik

Hi bednar ,
Actually, it's exactly the same - so you can use rvc3_support either from the host computer or when ssh-ed on the rvc3 itself. Regarding VIO, we are already implementing our own solution that will run on rvc3, and will likely integrate it into our SDK in the next couple of weeks. We will share updates here (on roadmap tag) 🙂
Thanks, Erik

bednar

Hi erik,

Thanks for the info. How to execute the rcv3 on the device, in exactly the same way? It is just not clear to me. Should the utilities/cam_test.py work? It actually did not work right away, but I might set it up wrong. Could you give me a hint? Should this guide work?

Thanks,
Jan

bednar

I have tried to run the utilities/cam_test.py

root@keembay:~/git/depthai-python-rvc3_support/utilities# python3 cam_test.py --cameras rgb,m

[2023-04-19 16:08:44.799] [warning] USB protocol not available - If running in a container, make sure that the following is set: "-v /dev/bus/usb:/dev/bus/usb --device-cgroup-rule='c 189:* rmw'"

Enabled cameras:

rgb : mono

DepthAI version: 2.21.2.0

DepthAI path: /home/root/.local/lib/python3.8/site-packages/depthai.cpython-38-aarch64-linux-gnu.so

Traceback (most recent call last):

File "cam_test.py", line 184, in <module>

with dai.Device(pipeline) as device:

RuntimeError: No available devices

root@keembay:~/git/depthai-python-rvc3_support/utilities#

I have plugged in a camera into the A port and I tried to run the code onboard the camera, not the host PC. What is the problem here? Could you direct me?

erik

Hi bednar ,
You have incorrect depthai version - in terminal it's 2.21.2, while you should have 2.19.x... - which is the one that gets downloaded when you are on rvc3_support and call python examples/install_requriemetns.py .You might need to copy over the library itself if you don't have internet connectivity on rvc3.
Thanks, Erik

bednar

Hi erik,

I have already tried that, but the install script examples/install_requirements.py installs the newest version 2.21.2. It seems to be wrong. Could you check it, please? I have downloaded the rvc3-support branch https://github.com/luxonis/depthai-python/tree/rvc3_support.

I have run the install_requiremenst.pyscript and manually installed the correct version with python3 -m pip install depthai==2.19.1.

I tried to run the utilities/cam_test.py script as before, and the result is the same.

root@keembay:~/git/depthai-python-rvc3_support/utilities# python3 cam_test.py --cameras rgb,m

[2023-04-20 10:37:58.290] [warning] USB protocol not available - If running in a container, make sure that the following is set: "-v /dev/bus/usb:/dev/bus/usb --device-cgroup-rule='c 189:* rmw'"

Enabled cameras:

rgb : mono

DepthAI version: 2.19.1.0

DepthAI path: /home/root/.local/lib/python3.8/site-packages/depthai.cpython-38-aarch64-linux-gnu.so

Traceback (most recent call last):

File "cam_test.py", line 184, in <module>

with dai.Device(pipeline) as device:

RuntimeError: No available devices

I also noticed, that there is an install script in utilites/install_requirements.py but this script does not work as below

root@keembay:~/git/depthai-python-rvc3_support/utilities# python3 install_requirements.py

Requirement already satisfied: pip in /home/root/.local/lib/python3.8/site-packages (23.1)

WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv

Requirement already satisfied: PySimpleGUI==4.60.3 in /home/root/.local/lib/python3.8/site-packages (from -r requirements.txt (line 1)) (4.60.3)

Requirement already satisfied: Pillow==9.2.0 in /home/root/.local/lib/python3.8/site-packages (from -r requirements.txt (line 2)) (9.2.0)

Traceback (most recent call last):

File "<string>", line 1, in <module>

File "/usr/lib/python3.8/tkinter/__init__.py", line 36, in <module>

import _tkinter # If this fails your Python may not be configured for Tk

ModuleNotFoundError: No module named '_tkinter'

Missing `python3-tk` on the system. Install with `sudo apt install python3-tk`

I tried to install the python3-tk but I do not have apt and I could not install it on the camera.

Now I tried to run the examples/ColorCamera/rgb_preview.py , and it still cannot see the cameras. I

root@keembay:~/git/depthai-python-rvc3_support/examples/ColorCamera# python3 rgb_preview.py

[2023-04-20 10:34:23.567] [warning] USB protocol not available - If running in a container, make sure that the following is set: "-v /dev/bus/usb:/dev/bus/usb --device-cgroup-rule='c 189:* rmw'"

Traceback (most recent call last):

File "rgb_preview.py", line 24, in <module>

with dai.Device(pipeline) as device:

RuntimeError: No available devices

Could you give me some hints, how to proceed?

Thanks!

bednar

One more interesting observation. I have updated the firwmare and OS to the newest version. Which one should I use? I followed the guide: https://docs.google.com/document/d/1r3en4EqgDG3hum8o5yS1DJ5vSCb6aNqyA38tIM8723U/edit#

Originally, I was testing the non-DepthAI functionality from the documentation so I ran these commands below

systemctl disable deviceservice

systemctl disable rm_vpu_daemon

reboot

I tried to revert those but they are not present anymore. Do you know what is the reason for that?

erik

Hi bednar ,
I'd suggest latest (1.8) OS version and fip-oak-ffc-6p-rvc3-1.2.0.bin FIP.
Thanks, Erik

erik

Regarding services - full access (via SSH) is allowed, but we do not officially support it, meaning we won't help when something goes wrong - as there are MANY ways to go wrong, and we would need 5 dedicated linux hackers to support everyone.
So in general, we suggest avoiding it altogether and use robothub for app deployment, which is the officially supported way to deploy apps to the RVC3. I hope you understand.

bednar

Hi erik,
sorry for the longer delay. I have updated the FIP and OS to the latest versions as attached in the images below.

I did not know that I should use only the Robothub! Thanks for that. Could you please state that somewhere in the start guide? I did not find it there. However, I tried to install the robothub on my PC to test it out with the camera connected to the PC. I failed with the installation as stated below. Could you tell me, what is the issue? I have treid to sudo apt-get update and sudo apt-get upgrade and no help.

Thank you!

bednar

I have been able to install it on my robot with Intel. However, the installation did not work on the AMD laptop as I showed in the images in the previous post.

erik

However, I cannot connect the OAK-FFC-6P device.

bednar Is that because there's no internet access from the rvc3 itself? You might want to look into that first. @StepanKroupa could you suggest on how to connect the OAK-FFC-6P to robothub?
Thanks, Erik

bednar

I have successfully connected my OAK-D-S2 FF to my RobotHub account. However, I cannot connect the OAK-FFC-6P device.

I have connected the OAK-FFC-6P to the robot and checked the USB device with lsusb and this device appeared, which seems to me wrong. I did not do anything on the device with the ssh anymore as you told me.

This is what dmesg -w tells me and it is still repeating the same error and cannot bind.

I tried to set UDEV rules, but it did not help.

Thanks for any advice!

bednar

erik I did not ssh to the drone after the flash, since I thought I should not do anything through the SSH. So should I set the internet connection as defined here https://docs.google.com/document/d/1r3en4EqgDG3hum8o5yS1DJ5vSCb6aNqyA38tIM8723U/edit and then use the RobotHub only?

StepanKroupa

Hello bednar , the RobotHub Agent comes pre-installed with LuxOS 1.8, however manually installing it as you have attempted might have corrupted the installation. Please flash the device with LuxOS 1.8 again, connect it to your LAN, then go to "https://IP:9010" (Agent's frontend) and click on the "Connect to RobotHub" button. If this doesn't work, try running robothub-ctl restart on the device to restart the Agent. With LuxOS 1.8 you might run into some issues, for example default Apps / example Apps will not work and video preview is disabled. These should be ironed out in the next release (1.9). For all Apps you create, please use this image. To do this, the robotapp.toml of your App should look something like this - you just need to change the link to the image on the very last line.

Please let us know if you have any issues with the setup!
Stepan K

bednar

Thanks for the support. Currently, I have to focus on other work. I will test it out and let you know later.

Kind Regards,
Jan