Example codes for RAE

garymok

I have just received my RAE. However, is there any example codes, for e.g., how to control LED, motors, etc.? Also, kickstarter campaign page mentioned that there will be mobile APP available for controlling it. When will it be available?

erik

Hi garymok ,
We have hardware docs here that have initial info on how to connect to the device, and we will be releasing demo repos tomorrow together with KS update. We apologize for the delay.

MatBee

erik Any luck on any documentation? The robothub examples don't work

erik

Hi MatBee ,
Sorry again, there was a slight delay, tomorrow some demos will get public and we will release KS update.

MatBee

Any github repos or branches I can look at while waiting for documentation?

erik

Hi MatBee @garymok ,
We have released rae-ros repo here: https://github.com/luxonis/rae-ros
It includes some files for ros driver along with some examples. I hope this helps!

garymok

erik

I have followed the example but encountered when building the docker image (on my Mac M1). I have to update the "--platform" parameter of docker build command to "linux/arm64" instead of "arm64". Otherwise, it will prompt error "failed to solve: no match for platform in manifest".

And then, the docker build process get stuck at step #7 with error as below:

Sometimes, the error is:

WARNING: experimental flag squash is removed with BuildKit. You should squash inside build using a multi-stage Dockerfile for efficiency.

[+] Building 38.4s (10/16) docker:desktop-linux

[+] Building 340.5s (11/16) docker:desktop-linux

=> [internal] load .dockerignore 0.0s

=> => transferring context: 2B 0.0s

=> [internal] load build definition from Dockerfile 0.0s

=> => transferring dockerfile: 1.49kB 0.0s

=> [internal] load metadata for docker.io/luxonis/depthai-ros-rae:latest 2.1s

=> [ 1/12] FROM docker.io/luxonis/depthai-ros-rae@sha256:5ac6773187d4564a28deed1e013b52a8349583dc8abfeea83220029bca278130 0.0s

=> => resolve docker.io/luxonis/depthai-ros-rae@sha256:5ac6773187d4564a28deed1e013b52a8349583dc8abfeea83220029bca278130 0.0s

=> [internal] load build context 0.0s

=> => transferring context: 12.77kB 0.0s

=> CACHED [ 2/12] RUN apt-get update && apt-get -y install --no-install-recommends software-properties-common git libusb-1.0-0-dev wget zsh python3-colcon-common-extensions python3-rosdep build-essen 0.0s

=> CACHED [ 3/12] RUN mkdir -p /ws/src 0.0s

=> CACHED [ 4/12] COPY ./ ./ws/src/rae-ros 0.0s

=> CACHED [ 5/12] RUN rm -rf ./ws/src/rae-ros/rae_gazebo 0.0s

=> CACHED [ 6/12] RUN cd ./ws/ && apt update && rosdep update && rosdep install --from-paths src --ignore-src -y --skip-keys depthai --skip-keys depthai_bridge --skip-keys depthai_ros_driver --skip-ke 0.0s

=> ERROR [ 7/12] RUN cd ./ws/ && . /opt/ros/humble/setup.sh && . /depthai_ws/install/setup.sh && colcon build --symlink-install --cmake-args -DCMAKE_BUILD_TYPE=RelWithDebInfo 338.2s

------

[ 7/12] RUN cd ./ws/ && . /opt/ros/humble/setup.sh && . /depthai_ws/install/setup.sh && colcon build --symlink-install --cmake-args -DCMAKE_BUILD_TYPE=RelWithDebInfo:

1.279 Starting >>> rae_msgs

1.291 Starting >>> rae_bringup

1.294 Starting >>> rae_camera

1.298 Starting >>> rae_description

2.296 Finished <<< rae_description [1.00s]

2.563 Finished <<< rae_bringup [1.27s]

4.855 Finished <<< rae_msgs [3.58s]

4.855 Starting >>> rae_hw

316.3 --- stderr: rae_camera

316.3 CMake Warning (dev) at CMakeLists.txt:67 (add_executable):

316.3 Policy CMP0115 is not set: Source file extensions must be explicit. Run

316.3 "cmake --help-policy CMP0115" for policy details. Use the cmake_policy

316.3 command to set the policy and suppress this warning.

316.3

316.3 File:

316.3

316.3 /ws/src/rae-ros/rae_camera/src/perception_ipc_rtabmap.cpp

316.3 This warning is for project developers. Use -Wno-dev to suppress it.

316.3

316.3 c++: fatal error: Killed signal terminated program cc1plus

316.3 compilation terminated.

316.3 gmake[2]: *** [CMakeFiles/yolo.dir/build.make:76: CMakeFiles/yolo.dir/src/yolo.cpp.o] Error 1

316.3 gmake[1]: *** [CMakeFiles/Makefile2:232: CMakeFiles/yolo.dir/all] Error 2

316.3 gmake[1]: *** Waiting for unfinished jobs....

316.3 /ws/src/rae-ros/rae_camera/src/stereo_pair.cpp: In function ‘int main()’:

316.3 /ws/src/rae-ros/rae_camera/src/stereo_pair.cpp:23:28: warning: ‘void dai::node::ColorCamera::setColorOrder(dai::ColorCameraProperties::ColorOrder)’ is deprecated: Use 'setPreviewType()' instead [-Wdeprecated-declarations]

316.3 23 | monoLeft->setColorOrder(dai::ColorCameraProperties::ColorOrder::BGR);

316.3 | ~~~~~~~~~~~~~~~~~~~~~~~^{~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~}

316.3 In file included from /usr/local/include/depthai/pipeline/nodes.hpp:6,

316.3 from /usr/local/include/depthai/depthai.hpp:17,

316.3 from /ws/src/rae-ros/rae_camera/src/stereo_pair.cpp:4:

316.3 /usr/local/include/depthai/pipeline/node/ColorCamera.hpp:140:59: note: declared here

316.3 140 | [[deprecated("Use 'setPreviewType()' instead")]] void setColorOrder(ColorCameraProperties::ColorOrder colorOrder);

316.3 | ^~~~~~~~~~~~~

316.3 /ws/src/rae-ros/rae_camera/src/stereo_depth.cpp: In function ‘int main()’:

316.3 /ws/src/rae-ros/rae_camera/src/stereo_depth.cpp:36:59: warning: ‘dai::CameraBoardSocket::LEFT’ is deprecated [-Wdeprecated-declarations]

316.3 36 | monoLeftFront->setBoardSocket(dai::CameraBoardSocket::LEFT);

316.3 | ^~~~

316.3 In file included from /usr/local/include/depthai/device/CalibrationHandler.hpp:5,

316.3 from /usr/local/include/depthai/depthai.hpp:9,

316.3 from /ws/src/rae-ros/rae_camera/src/stereo_depth.cpp:4:

316.3 /usr/local/include/depthai-shared/common/CameraBoardSocket.hpp:25:5: note: declared here

316.3 25 | LEFT [[deprecated]] = CAM_B,

316.3 | ^~~~

316.3 /ws/src/rae-ros/rae_camera/src/stereo_depth.cpp:36:59: warning: ‘dai::CameraBoardSocket::LEFT’ is deprecated [-Wdeprecated-declarations]

316.3 36 | monoLeftFront->setBoardSocket(dai::CameraBoardSocket::LEFT);

316.3 | ^~~~

316.3 In file included from /usr/local/include/depthai/device/CalibrationHandler.hpp:5,

316.3 from /usr/local/include/depthai/depthai.hpp:9,

316.3 from /ws/src/rae-ros/rae_camera/src/stereo_depth.cpp:4:

316.3 /usr/local/include/depthai-shared/common/CameraBoardSocket.hpp:25:5: note: declared here

316.3 25 | LEFT [[deprecated]] = CAM_B,

316.3 | ^~~~

316.3 /ws/src/rae-ros/rae_camera/src/stereo_depth.cpp:38:60: warning: ‘dai::CameraBoardSocket::RIGHT’ is deprecated [-Wdeprecated-declarations]

316.3 38 | monoRightFront->setBoardSocket(dai::CameraBoardSocket::RIGHT);

316.3 | ^~~~~

316.3 In file included from /usr/local/include/depthai/device/CalibrationHandler.hpp:5,

316.3 from /usr/local/include/depthai/depthai.hpp:9,

316.3 from /ws/src/rae-ros/rae_camera/src/stereo_depth.cpp:4:

316.3 /usr/local/include/depthai-shared/common/CameraBoardSocket.hpp:26:5: note: declared here

316.3 26 | RIGHT [[deprecated]] = CAM_C,

316.3 | ^~~~~

316.3 /ws/src/rae-ros/rae_camera/src/stereo_depth.cpp:38:60: warning: ‘dai::CameraBoardSocket::RIGHT’ is deprecated [-Wdeprecated-declarations]

316.3 38 | monoRightFront->setBoardSocket(dai::CameraBoardSocket::RIGHT);

316.3 | ^~~~~

316.3 In file included from /usr/local/include/depthai/device/CalibrationHandler.hpp:5,

316.3 from /usr/local/include/depthai/depthai.hpp:9,

316.3 from /ws/src/rae-ros/rae_camera/src/stereo_depth.cpp:4:

316.3 /usr/local/include/depthai-shared/common/CameraBoardSocket.hpp:26:5: note: declared here

316.3 26 | RIGHT [[deprecated]] = CAM_C,

316.3 | ^~~~~

316.3 /ws/src/rae-ros/rae_camera/src/camera.cpp: In function ‘dai:😛ipeline createPipeline(bool, bool)’:

316.3 /ws/src/rae-ros/rae_camera/src/camera.cpp:29:56: warning: ‘dai::CameraBoardSocket::RGB’ is deprecated [-Wdeprecated-declarations]

316.3 29 | camRgb->setBoardSocket(dai::CameraBoardSocket::RGB);

316.3 | ^~~

316.3 In file included from /usr/local/include/depthai/common/CameraBoardSocket.hpp:5,

316.3 from /usr/local/include/depthai/device/DeviceBase.hpp:18,

316.3 from /usr/local/include/depthai/device/Device.hpp:14,

316.3 from /ws/src/rae-ros/rae_camera/src/camera.cpp:13:

316.3 /usr/local/include/depthai-shared/common/CameraBoardSocket.hpp:23:5: note: declared here

316.3 23 | RGB [[deprecated]] = CAM_A,

316.3 | ^~~

316.3 /ws/src/rae-ros/rae_camera/src/camera.cpp:29:56: warning: ‘dai::CameraBoardSocket::RGB’ is deprecated [-Wdeprecated-declarations]

316.3 29 | camRgb->setBoardSocket(dai::CameraBoardSocket::RGB);

316.3 | ^~~

316.3 In file included from /usr/local/include/depthai/common/CameraBoardSocket.hpp:5,

316.3 from /usr/local/include/depthai/device/DeviceBase.hpp:18,

316.3 from /usr/local/include/depthai/device/Device.hpp:14,

316.3 from /ws/src/rae-ros/rae_camera/src/camera.cpp:13:

316.3 /usr/local/include/depthai-shared/common/CameraBoardSocket.hpp:23:5: note: declared here

316.3 23 | RGB [[deprecated]] = CAM_A,

316.3 | ^~~

316.3 /ws/src/rae-ros/rae_camera/src/camera.cpp:30:30: warning: ‘void dai::node::ColorCamera::setColorOrder(dai::ColorCameraProperties::ColorOrder)’ is deprecated: Use 'setPreviewType()' instead [-Wdeprecated-declarations]

316.3 30 | camRgb->setColorOrder(dai::ColorCameraProperties::ColorOrder::BGR);

316.3 | ~~~~~~~~~~~~~~~~~~~~~^{~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~}

316.3 In file included from /ws/src/rae-ros/rae_camera/src/camera.cpp:15:

316.3 /usr/local/include/depthai/pipeline/node/ColorCamera.hpp:140:59: note: declared here

316.3 140 | [[deprecated("Use 'setPreviewType()' instead")]] void setColorOrder(ColorCameraProperties::ColorOrder colorOrder);

316.3 | ^~~~~~~~~~~~~

316.3 /ws/src/rae-ros/rae_camera/src/camera.cpp:31:31: warning: ‘void dai::node::ColorCamera::setInterleaved(bool)’ is deprecated: Use 'setPreviewType()' instead [-Wdeprecated-declarations]

316.3 31 | camRgb->setInterleaved(false);

316.3 | ~~~~~~~~~~~~~~~~~~~~~~^~~~~~~

316.3 In file included from /ws/src/rae-ros/rae_camera/src/camera.cpp:15:

316.3 /usr/local/include/depthai/pipeline/node/ColorCamera.hpp:146:59: note: declared here

316.3 146 | [[deprecated("Use 'setPreviewType()' instead")]] void setInterleaved(bool interleaved);

316.3 | ^{~~~~~~~~~~~~~}

316.3 /ws/src/rae-ros/rae_camera/src/camera.cpp:58:55: warning: ‘dai::CameraBoardSocket::LEFT’ is deprecated [-Wdeprecated-declarations]

316.3 58 | leftFront->setBoardSocket(dai::CameraBoardSocket::LEFT);

316.3 | ^~~~

316.3 In file included from /usr/local/include/depthai/common/CameraBoardSocket.hpp:5,

316.3 from /usr/local/include/depthai/device/DeviceBase.hpp:18,

316.3 from /usr/local/include/depthai/device/Device.hpp:14,

316.3 from /ws/src/rae-ros/rae_camera/src/camera.cpp:13:

316.3 /usr/local/include/depthai-shared/common/CameraBoardSocket.hpp:25:5: note: declared here

316.3 25 | LEFT [[deprecated]] = CAM_B,

316.3 | ^~~~

316.3 /ws/src/rae-ros/rae_camera/src/camera.cpp:58:55: warning: ‘dai::CameraBoardSocket::LEFT’ is deprecated [-Wdeprecated-declarations]

316.3 58 | leftFront->setBoardSocket(dai::CameraBoardSocket::LEFT);

316.3 | ^~~~

316.3 In file included from /usr/local/include/depthai/common/CameraBoardSocket.hpp:5,

316.3 from /usr/local/include/depthai/device/DeviceBase.hpp:18,

316.3 from /usr/local/include/depthai/device/Device.hpp:14,

316.3 from /ws/src/rae-ros/rae_camera/src/camera.cpp:13:

316.3 /usr/local/include/depthai-shared/common/CameraBoardSocket.hpp:25:5: note: declared here

316.3 25 | LEFT [[deprecated]] = CAM_B,

316.3 | ^~~~

316.3 /ws/src/rae-ros/rae_camera/src/camera.cpp:60:56: warning: ‘dai::CameraBoardSocket::RIGHT’ is deprecated [-Wdeprecated-declarations]

316.3 60 | rightFront->setBoardSocket(dai::CameraBoardSocket::RIGHT);

316.3 | ^~~~~

316.3 In file included from /usr/local/include/depthai/common/CameraBoardSocket.hpp:5,

316.3 from /usr/local/include/depthai/device/DeviceBase.hpp:18,

316.3 from /usr/local/include/depthai/device/Device.hpp:14,

316.3 from /ws/src/rae-ros/rae_camera/src/camera.cpp:13:

316.3 /usr/local/include/depthai-shared/common/CameraBoardSocket.hpp:26:5: note: declared here

316.3 26 | RIGHT [[deprecated]] = CAM_C,

316.3 | ^~~~~

316.3 /ws/src/rae-ros/rae_camera/src/camera.cpp:60:56: warning: ‘dai::CameraBoardSocket::RIGHT’ is deprecated [-Wdeprecated-declarations]

316.3 60 | rightFront->setBoardSocket(dai::CameraBoardSocket::RIGHT);

316.3 | ^~~~~

316.3 In file included from /usr/local/include/depthai/common/CameraBoardSocket.hpp:5,

316.3 from /usr/local/include/depthai/device/DeviceBase.hpp:18,

316.3 from /usr/local/include/depthai/device/Device.hpp:14,

316.3 from /ws/src/rae-ros/rae_camera/src/camera.cpp:13:

316.3 /usr/local/include/depthai-shared/common/CameraBoardSocket.hpp:26:5: note: declared here

316.3 26 | RIGHT [[deprecated]] = CAM_C,

316.3 | ^~~~~

316.3 /ws/src/rae-ros/rae_camera/src/camera.cpp: In function ‘int main(int, char**)’:

316.3 /ws/src/rae-ros/rae_camera/src/camera.cpp:198:135: warning: ‘dai::CameraBoardSocket::RIGHT’ is deprecated [-Wdeprecated-declarations]

316.3 198 | sensor_msgs::msg::CameraInfo rgbCameraInfo = rgbConverter.calibrationToCameraInfo(calibrationHandler, dai::CameraBoardSocket::RIGHT, width, height);;

316.3 | ^~~~~

316.3 In file included from /usr/local/include/depthai/common/CameraBoardSocket.hpp:5,

316.3 from /usr/local/include/depthai/device/DeviceBase.hpp:18,

316.3 from /usr/local/include/depthai/device/Device.hpp:14,

316.3 from /ws/src/rae-ros/rae_camera/src/camera.cpp:13:

316.3 /usr/local/include/depthai-shared/common/CameraBoardSocket.hpp:26:5: note: declared here

316.3 26 | RIGHT [[deprecated]] = CAM_C,

316.3 | ^~~~~

316.3 /ws/src/rae-ros/rae_camera/src/camera.cpp:198:135: warning: ‘dai::CameraBoardSocket::RIGHT’ is deprecated [-Wdeprecated-declarations]

316.3 198 | sensor_msgs::msg::CameraInfo rgbCameraInfo = rgbConverter.calibrationToCameraInfo(calibrationHandler, dai::CameraBoardSocket::RIGHT, width, height);;

316.3 | ^~~~~

316.3 In file included from /usr/local/include/depthai/common/CameraBoardSocket.hpp:5,

316.3 from /usr/local/include/depthai/device/DeviceBase.hpp:18,

316.3 from /usr/local/include/depthai/device/Device.hpp:14,

316.3 from /ws/src/rae-ros/rae_camera/src/camera.cpp:13:

316.3 /usr/local/include/depthai-shared/common/CameraBoardSocket.hpp:26:5: note: declared here

316.3 26 | RIGHT [[deprecated]] = CAM_C,

316.3 | ^~~~~

316.3 c++: fatal error: Killed signal terminated program cc1plus

316.3 compilation terminated.

316.3 gmake[2]: *** [CMakeFiles/stereo_pair.dir/build.make:76: CMakeFiles/stereo_pair.dir/src/stereo_pair.cpp.o] Error 1

316.3 gmake[1]: *** [CMakeFiles/Makefile2:258: CMakeFiles/stereo_pair.dir/all] Error 2

316.3 gmake: *** [Makefile:146: all] Error 2

316.3 ---

316.3 Failed <<< rae_camera [5min 15s, exited with code 2]

337.7 [Processing: rae_camera, rae_hw]

337.7 Aborted <<< rae_hw [5min 33s]

337.9

337.9 Summary: 3 packages finished [5min 37s]

337.9 1 package failed: rae_camera

337.9 1 package aborted: rae_hw

337.9 2 packages had stderr output: rae_camera rae_hw

------

Dockerfile:19

--------------------

17 | RUN cd .$WS/ && apt update && rosdep update && rosdep install --from-paths src --ignore-src -y --skip-keys depthai --skip-keys depthai_bridge --skip-keys depthai_ros_driver --skip-keys audio_msgs --skip-keys laserscan_kinect --skip-keys ira_laser_tools

18 |

19 | >>> RUN cd .$WS/ && . /opt/ros/${ROS_DISTRO}/setup.sh && . $UNDERLAY_WS/install/setup.sh && colcon build --symlink-install --cmake-args -DCMAKE_BUILD_TYPE=${BUILD_TYPE}

20 |

21 | RUN echo "if [ -f ${WS}/install/setup.zsh ]; then source ${WS}/install/setup.zsh; fi" >> $HOME/.zshrc

--------------------

ERROR: failed to solve: process "/bin/sh -c cd .$WS/ && . /opt/ros/${ROS_DISTRO}/setup.sh && . $UNDERLAY_WS/install/setup.sh && colcon build --symlink-install --cmake-args -DCMAKE_BUILD_TYPE=${BUILD_TYPE}" did not complete successfully: exit code: 2

MatBee

garymok You didn't even post the command youre trying to use to build the Dockerfile- the project builds just fine for me.

They may have merged changes since then, however

MatBee

garymok apologies, they merged some PR's since and it's currently broken. Their repo should be ignored for now, I think. Pack away the rae, it's no use atm.

MatBee

garymok Check the repo again, just use their prebuilt image on docker hub

erik

@garymok Could you post an issue of this error on the repo itself? This would get you in contact with ros dev team directly.

Wijay

Just received my rae. Trying to get it on to the internet. Error at the second step is:

root@keembay:/etc# wpa_supplicant -B -i wlp1s0 -c /etc/wpa_supplicant.conf

Successfully initialized wpa_supplicant

nl80211: kernel reports: Match already configured

nl80211: Could not configure driver mode

nl80211: deinit ifname=wlp1s0 disabled_11b_rates=0

wlp1s0: Failed to initialize driver interface

reset rae and tried again. No luck. It is not able to join the network and get a DHCP address. What could I try next?

Also noticed that after the first login via wifi, I have only been able to login via usb.

Tx.

erik

Hi Wijay ,
Could you try following Robothub installation? That will also configure the wifi before connecting to the robothub cloud.

Wijay

erik Hi Erik, I assume you are referring to the method where you scan the QR code? Tried that first and it does not work.

geekscape

hi @Wijay, I found that I needed to enter the command wpa_supplicant -B -i wlp1s0 -c /etc/wpa_supplicant.conf twice (fails the first time).

The following sequence of commands works for my rae …
systemctl stop systemd-networkd
wpa_supplicant -B -i wlp1s0 -c /etc/wpa_supplicant.conf
ps ax | grep wpa_supplicant # If process isn't running, try again
wpa_supplicant -B -i wlp1s0 -c /etc/wpa_supplicant.conf
systemctl status systemd-networkd # Network daemon should have restarted automatically
ifconfig wlp1s0 # Interface should be up and have an IP address

Wijay

geekscape Ji geekscape

Thanks. Running the wpa_supplicant -B -i wlp1s0 -c /etc/wpa_supplicant.conf command the second time works.

Many thanks. Let me proceed and see how I go.

geekscape

hi @erik, whilst the Luxonis RobotHub looks useful … it is a good thing for the rae (and all other products) to be able to operate independently of RobotHub in other cases. After all, it is running Linux … and Wi-Fi should just work 🙂

I haven't figured out yet why the wpa_supplicant process is failing on my rae … the first time that the wpa_supplicant -B -i wlp1s0 -c /etc/wpa_supplicant.conf command is entered and then succeeds the second time, but will diagnose further.

Regarding using the Luxonis RobotHub to configure Wi-Fi on the rae robot … entering one's personal Wi-Fi network SSID and password on a third-party web-site is a security risk, which some people may be comfortable with, but really is a poor security practice that should be avoided. At the very least, offering an open-source tool that can be reviewed and run locally to produce a QR-code on the desktop would be better … thanks !

Note: I realize that completing a KickStarter hardware project … including all the software, firmware and documentation is a huge endeavor. Please don't take my comments as being overly critical … I like the Luxonis products, as well as the commitment to open-source.

loxscs

geekscape @erik

Something that could be added to docs would be to use wpa_passphrase <ssid> to generate part of the wpa_supplicant.conf file and not have the password in plaintext anywhere by not copying the #psk= portion of the output.

Wijay

Stupid question. Where is the remote control app for rae? Or is it served as an http page on rae itself?

Thanks.

Wijay

Sorry. Figured out that it is <ip address of rae>:9010.