• RAE

  • Example codes for RAE

I have just received my RAE. However, is there any example codes, for e.g., how to control LED, motors, etc.? Also, kickstarter campaign page mentioned that there will be mobile APP available for controlling it. When will it be available?

  • erik replied to this.

    Hi garymok ,
    We have hardware docs here that have initial info on how to connect to the device, and we will be releasing demo repos tomorrow together with KS update. We apologize for the delay.

        erik Any luck on any documentation? The robothub examples don't work

        • erik replied to this.

            Hi MatBee ,
            Sorry again, there was a slight delay, tomorrow some demos will get public and we will release KS update.

              Any github repos or branches I can look at while waiting for documentation?

              • erik replied to this.

                erik

                I have followed the example but encountered when building the docker image (on my Mac M1). I have to update the "--platform" parameter of docker build command to "linux/arm64" instead of "arm64". Otherwise, it will prompt error "failed to solve: no match for platform in manifest".

                And then, the docker build process get stuck at step #7 with error as below:

                Sometimes, the error is:

                WARNING: experimental flag squash is removed with BuildKit. You should squash inside build using a multi-stage Dockerfile for efficiency.

                [+] Building 38.4s (10/16) docker:desktop-linux

                [+] Building 340.5s (11/16) docker:desktop-linux

                => [internal] load .dockerignore 0.0s

                => => transferring context: 2B 0.0s

                => [internal] load build definition from Dockerfile 0.0s

                => => transferring dockerfile: 1.49kB 0.0s

                => [internal] load metadata for docker.io/luxonis/depthai-ros-rae:latest 2.1s

                => [ 1/12] FROM docker.io/luxonis/depthai-ros-rae@sha256:5ac6773187d4564a28deed1e013b52a8349583dc8abfeea83220029bca278130 0.0s

                => => resolve docker.io/luxonis/depthai-ros-rae@sha256:5ac6773187d4564a28deed1e013b52a8349583dc8abfeea83220029bca278130 0.0s

                => [internal] load build context 0.0s

                => => transferring context: 12.77kB 0.0s

                => CACHED [ 2/12] RUN apt-get update && apt-get -y install --no-install-recommends software-properties-common git libusb-1.0-0-dev wget zsh python3-colcon-common-extensions python3-rosdep build-essen 0.0s

                => CACHED [ 3/12] RUN mkdir -p /ws/src 0.0s

                => CACHED [ 4/12] COPY ./ ./ws/src/rae-ros 0.0s

                => CACHED [ 5/12] RUN rm -rf ./ws/src/rae-ros/rae_gazebo 0.0s

                => CACHED [ 6/12] RUN cd ./ws/ && apt update && rosdep update && rosdep install --from-paths src --ignore-src -y --skip-keys depthai --skip-keys depthai_bridge --skip-keys depthai_ros_driver --skip-ke 0.0s

                => ERROR [ 7/12] RUN cd ./ws/ && . /opt/ros/humble/setup.sh && . /depthai_ws/install/setup.sh && colcon build --symlink-install --cmake-args -DCMAKE_BUILD_TYPE=RelWithDebInfo 338.2s

                ------

                [ 7/12] RUN cd ./ws/ && . /opt/ros/humble/setup.sh && . /depthai_ws/install/setup.sh && colcon build --symlink-install --cmake-args -DCMAKE_BUILD_TYPE=RelWithDebInfo:

                1.279 Starting >>> rae_msgs

                1.291 Starting >>> rae_bringup

                1.294 Starting >>> rae_camera

                1.298 Starting >>> rae_description

                2.296 Finished <<< rae_description [1.00s]

                2.563 Finished <<< rae_bringup [1.27s]

                4.855 Finished <<< rae_msgs [3.58s]

                4.855 Starting >>> rae_hw

                316.3 --- stderr: rae_camera

                316.3 CMake Warning (dev) at CMakeLists.txt:67 (add_executable):

                316.3 Policy CMP0115 is not set: Source file extensions must be explicit. Run

                316.3 "cmake --help-policy CMP0115" for policy details. Use the cmake_policy

                316.3 command to set the policy and suppress this warning.

                316.3

                316.3 File:

                316.3

                316.3 /ws/src/rae-ros/rae_camera/src/perception_ipc_rtabmap.cpp

                316.3 This warning is for project developers. Use -Wno-dev to suppress it.

                316.3

                316.3 c++: fatal error: Killed signal terminated program cc1plus

                316.3 compilation terminated.

                316.3 gmake[2]: *** [CMakeFiles/yolo.dir/build.make:76: CMakeFiles/yolo.dir/src/yolo.cpp.o] Error 1

                316.3 gmake[1]: *** [CMakeFiles/Makefile2:232: CMakeFiles/yolo.dir/all] Error 2

                316.3 gmake[1]: *** Waiting for unfinished jobs....

                316.3 /ws/src/rae-ros/rae_camera/src/stereo_pair.cpp: In function ‘int main()’:

                316.3 /ws/src/rae-ros/rae_camera/src/stereo_pair.cpp:23:28: warning: ‘void dai::node::ColorCamera::setColorOrder(dai::ColorCameraProperties::ColorOrder)’ is deprecated: Use 'setPreviewType()' instead [-Wdeprecated-declarations]

                316.3 23 | monoLeft->setColorOrder(dai::ColorCameraProperties::ColorOrder::BGR);

                316.3 | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

                316.3 In file included from /usr/local/include/depthai/pipeline/nodes.hpp:6,

                316.3 from /usr/local/include/depthai/depthai.hpp:17,

                316.3 from /ws/src/rae-ros/rae_camera/src/stereo_pair.cpp:4:

                316.3 /usr/local/include/depthai/pipeline/node/ColorCamera.hpp:140:59: note: declared here

                316.3 140 | [[deprecated("Use 'setPreviewType()' instead")]] void setColorOrder(ColorCameraProperties::ColorOrder colorOrder);

                316.3 | ~~~~~~~~~~~~

                316.3 /ws/src/rae-ros/rae_camera/src/stereo_depth.cpp: In function ‘int main()’:

                316.3 /ws/src/rae-ros/rae_camera/src/stereo_depth.cpp:36:59: warning: ‘dai::CameraBoardSocket::LEFT’ is deprecated [-Wdeprecated-declarations]

                316.3 36 | monoLeftFront->setBoardSocket(dai::CameraBoardSocket::LEFT);

                316.3 | ~~~

                316.3 In file included from /usr/local/include/depthai/device/CalibrationHandler.hpp:5,

                316.3 from /usr/local/include/depthai/depthai.hpp:9,

                316.3 from /ws/src/rae-ros/rae_camera/src/stereo_depth.cpp:4:

                316.3 /usr/local/include/depthai-shared/common/CameraBoardSocket.hpp:25:5: note: declared here

                316.3 25 | LEFT [[deprecated]] = CAM_B,

                316.3 | ~~~

                316.3 /ws/src/rae-ros/rae_camera/src/stereo_depth.cpp:36:59: warning: ‘dai::CameraBoardSocket::LEFT’ is deprecated [-Wdeprecated-declarations]

                316.3 36 | monoLeftFront->setBoardSocket(dai::CameraBoardSocket::LEFT);

                316.3 | ~~~

                316.3 In file included from /usr/local/include/depthai/device/CalibrationHandler.hpp:5,

                316.3 from /usr/local/include/depthai/depthai.hpp:9,

                316.3 from /ws/src/rae-ros/rae_camera/src/stereo_depth.cpp:4:

                316.3 /usr/local/include/depthai-shared/common/CameraBoardSocket.hpp:25:5: note: declared here

                316.3 25 | LEFT [[deprecated]] = CAM_B,

                316.3 | ~~~

                316.3 /ws/src/rae-ros/rae_camera/src/stereo_depth.cpp:38:60: warning: ‘dai::CameraBoardSocket::RIGHT’ is deprecated [-Wdeprecated-declarations]

                316.3 38 | monoRightFront->setBoardSocket(dai::CameraBoardSocket::RIGHT);

                316.3 | ~~~~

                316.3 In file included from /usr/local/include/depthai/device/CalibrationHandler.hpp:5,

                316.3 from /usr/local/include/depthai/depthai.hpp:9,

                316.3 from /ws/src/rae-ros/rae_camera/src/stereo_depth.cpp:4:

                316.3 /usr/local/include/depthai-shared/common/CameraBoardSocket.hpp:26:5: note: declared here

                316.3 26 | RIGHT [[deprecated]] = CAM_C,

                316.3 | ~~~~

                316.3 /ws/src/rae-ros/rae_camera/src/stereo_depth.cpp:38:60: warning: ‘dai::CameraBoardSocket::RIGHT’ is deprecated [-Wdeprecated-declarations]

                316.3 38 | monoRightFront->setBoardSocket(dai::CameraBoardSocket::RIGHT);

                316.3 | ~~~~

                316.3 In file included from /usr/local/include/depthai/device/CalibrationHandler.hpp:5,

                316.3 from /usr/local/include/depthai/depthai.hpp:9,

                316.3 from /ws/src/rae-ros/rae_camera/src/stereo_depth.cpp:4:

                316.3 /usr/local/include/depthai-shared/common/CameraBoardSocket.hpp:26:5: note: declared here

                316.3 26 | RIGHT [[deprecated]] = CAM_C,

                316.3 | ~~~~

                316.3 /ws/src/rae-ros/rae_camera/src/camera.cpp: In function ‘dai:😛ipeline createPipeline(bool, bool)’:

                316.3 /ws/src/rae-ros/rae_camera/src/camera.cpp:29:56: warning: ‘dai::CameraBoardSocket::RGB’ is deprecated [-Wdeprecated-declarations]

                316.3 29 | camRgb->setBoardSocket(dai::CameraBoardSocket::RGB);

                316.3 | ~~

                316.3 In file included from /usr/local/include/depthai/common/CameraBoardSocket.hpp:5,

                316.3 from /usr/local/include/depthai/device/DeviceBase.hpp:18,

                316.3 from /usr/local/include/depthai/device/Device.hpp:14,

                316.3 from /ws/src/rae-ros/rae_camera/src/camera.cpp:13:

                316.3 /usr/local/include/depthai-shared/common/CameraBoardSocket.hpp:23:5: note: declared here

                316.3 23 | RGB [[deprecated]] = CAM_A,

                316.3 | ~~

                316.3 /ws/src/rae-ros/rae_camera/src/camera.cpp:29:56: warning: ‘dai::CameraBoardSocket::RGB’ is deprecated [-Wdeprecated-declarations]

                316.3 29 | camRgb->setBoardSocket(dai::CameraBoardSocket::RGB);

                316.3 | ~~

                316.3 In file included from /usr/local/include/depthai/common/CameraBoardSocket.hpp:5,

                316.3 from /usr/local/include/depthai/device/DeviceBase.hpp:18,

                316.3 from /usr/local/include/depthai/device/Device.hpp:14,

                316.3 from /ws/src/rae-ros/rae_camera/src/camera.cpp:13:

                316.3 /usr/local/include/depthai-shared/common/CameraBoardSocket.hpp:23:5: note: declared here

                316.3 23 | RGB [[deprecated]] = CAM_A,

                316.3 | ~~

                316.3 /ws/src/rae-ros/rae_camera/src/camera.cpp:30:30: warning: ‘void dai::node::ColorCamera::setColorOrder(dai::ColorCameraProperties::ColorOrder)’ is deprecated: Use 'setPreviewType()' instead [-Wdeprecated-declarations]

                316.3 30 | camRgb->setColorOrder(dai::ColorCameraProperties::ColorOrder::BGR);

                316.3 | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

                316.3 In file included from /ws/src/rae-ros/rae_camera/src/camera.cpp:15:

                316.3 /usr/local/include/depthai/pipeline/node/ColorCamera.hpp:140:59: note: declared here

                316.3 140 | [[deprecated("Use 'setPreviewType()' instead")]] void setColorOrder(ColorCameraProperties::ColorOrder colorOrder);

                316.3 | ~~~~~~~~~~~~

                316.3 /ws/src/rae-ros/rae_camera/src/camera.cpp:31:31: warning: ‘void dai::node::ColorCamera::setInterleaved(bool)’ is deprecated: Use 'setPreviewType()' instead [-Wdeprecated-declarations]

                316.3 31 | camRgb->setInterleaved(false);

                316.3 | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~

                316.3 In file included from /ws/src/rae-ros/rae_camera/src/camera.cpp:15:

                316.3 /usr/local/include/depthai/pipeline/node/ColorCamera.hpp:146:59: note: declared here

                316.3 146 | [[deprecated("Use 'setPreviewType()' instead")]] void setInterleaved(bool interleaved);

                316.3 | ~~~~~~~~~~~~~

                316.3 /ws/src/rae-ros/rae_camera/src/camera.cpp:58:55: warning: ‘dai::CameraBoardSocket::LEFT’ is deprecated [-Wdeprecated-declarations]

                316.3 58 | leftFront->setBoardSocket(dai::CameraBoardSocket::LEFT);

                316.3 | ~~~

                316.3 In file included from /usr/local/include/depthai/common/CameraBoardSocket.hpp:5,

                316.3 from /usr/local/include/depthai/device/DeviceBase.hpp:18,

                316.3 from /usr/local/include/depthai/device/Device.hpp:14,

                316.3 from /ws/src/rae-ros/rae_camera/src/camera.cpp:13:

                316.3 /usr/local/include/depthai-shared/common/CameraBoardSocket.hpp:25:5: note: declared here

                316.3 25 | LEFT [[deprecated]] = CAM_B,

                316.3 | ~~~

                316.3 /ws/src/rae-ros/rae_camera/src/camera.cpp:58:55: warning: ‘dai::CameraBoardSocket::LEFT’ is deprecated [-Wdeprecated-declarations]

                316.3 58 | leftFront->setBoardSocket(dai::CameraBoardSocket::LEFT);

                316.3 | ~~~

                316.3 In file included from /usr/local/include/depthai/common/CameraBoardSocket.hpp:5,

                316.3 from /usr/local/include/depthai/device/DeviceBase.hpp:18,

                316.3 from /usr/local/include/depthai/device/Device.hpp:14,

                316.3 from /ws/src/rae-ros/rae_camera/src/camera.cpp:13:

                316.3 /usr/local/include/depthai-shared/common/CameraBoardSocket.hpp:25:5: note: declared here

                316.3 25 | LEFT [[deprecated]] = CAM_B,

                316.3 | ~~~

                316.3 /ws/src/rae-ros/rae_camera/src/camera.cpp:60:56: warning: ‘dai::CameraBoardSocket::RIGHT’ is deprecated [-Wdeprecated-declarations]

                316.3 60 | rightFront->setBoardSocket(dai::CameraBoardSocket::RIGHT);

                316.3 | ~~~~

                316.3 In file included from /usr/local/include/depthai/common/CameraBoardSocket.hpp:5,

                316.3 from /usr/local/include/depthai/device/DeviceBase.hpp:18,

                316.3 from /usr/local/include/depthai/device/Device.hpp:14,

                316.3 from /ws/src/rae-ros/rae_camera/src/camera.cpp:13:

                316.3 /usr/local/include/depthai-shared/common/CameraBoardSocket.hpp:26:5: note: declared here

                316.3 26 | RIGHT [[deprecated]] = CAM_C,

                316.3 | ~~~~

                316.3 /ws/src/rae-ros/rae_camera/src/camera.cpp:60:56: warning: ‘dai::CameraBoardSocket::RIGHT’ is deprecated [-Wdeprecated-declarations]

                316.3 60 | rightFront->setBoardSocket(dai::CameraBoardSocket::RIGHT);

                316.3 | ~~~~

                316.3 In file included from /usr/local/include/depthai/common/CameraBoardSocket.hpp:5,

                316.3 from /usr/local/include/depthai/device/DeviceBase.hpp:18,

                316.3 from /usr/local/include/depthai/device/Device.hpp:14,

                316.3 from /ws/src/rae-ros/rae_camera/src/camera.cpp:13:

                316.3 /usr/local/include/depthai-shared/common/CameraBoardSocket.hpp:26:5: note: declared here

                316.3 26 | RIGHT [[deprecated]] = CAM_C,

                316.3 | ~~~~

                316.3 /ws/src/rae-ros/rae_camera/src/camera.cpp: In function ‘int main(int, char**)’:

                316.3 /ws/src/rae-ros/rae_camera/src/camera.cpp:198:135: warning: ‘dai::CameraBoardSocket::RIGHT’ is deprecated [-Wdeprecated-declarations]

                316.3 198 | sensor_msgs::msg::CameraInfo rgbCameraInfo = rgbConverter.calibrationToCameraInfo(calibrationHandler, dai::CameraBoardSocket::RIGHT, width, height);;

                316.3 | ~~~~

                316.3 In file included from /usr/local/include/depthai/common/CameraBoardSocket.hpp:5,

                316.3 from /usr/local/include/depthai/device/DeviceBase.hpp:18,

                316.3 from /usr/local/include/depthai/device/Device.hpp:14,

                316.3 from /ws/src/rae-ros/rae_camera/src/camera.cpp:13:

                316.3 /usr/local/include/depthai-shared/common/CameraBoardSocket.hpp:26:5: note: declared here

                316.3 26 | RIGHT [[deprecated]] = CAM_C,

                316.3 | ~~~~

                316.3 /ws/src/rae-ros/rae_camera/src/camera.cpp:198:135: warning: ‘dai::CameraBoardSocket::RIGHT’ is deprecated [-Wdeprecated-declarations]

                316.3 198 | sensor_msgs::msg::CameraInfo rgbCameraInfo = rgbConverter.calibrationToCameraInfo(calibrationHandler, dai::CameraBoardSocket::RIGHT, width, height);;

                316.3 | ~~~~

                316.3 In file included from /usr/local/include/depthai/common/CameraBoardSocket.hpp:5,

                316.3 from /usr/local/include/depthai/device/DeviceBase.hpp:18,

                316.3 from /usr/local/include/depthai/device/Device.hpp:14,

                316.3 from /ws/src/rae-ros/rae_camera/src/camera.cpp:13:

                316.3 /usr/local/include/depthai-shared/common/CameraBoardSocket.hpp:26:5: note: declared here

                316.3 26 | RIGHT [[deprecated]] = CAM_C,

                316.3 | ~~~~

                316.3 c++: fatal error: Killed signal terminated program cc1plus

                316.3 compilation terminated.

                316.3 gmake[2]: *** [CMakeFiles/stereo_pair.dir/build.make:76: CMakeFiles/stereo_pair.dir/src/stereo_pair.cpp.o] Error 1

                316.3 gmake[1]: *** [CMakeFiles/Makefile2:258: CMakeFiles/stereo_pair.dir/all] Error 2

                316.3 gmake: *** [Makefile:146: all] Error 2

                316.3 ---

                316.3 Failed <<< rae_camera [5min 15s, exited with code 2]

                337.7 [Processing: rae_camera, rae_hw]

                337.7 [Processing: rae_camera, rae_hw]

                337.7 [Processing: rae_camera, rae_hw]

                337.7 [Processing: rae_camera, rae_hw]

                337.7 [Processing: rae_camera, rae_hw]

                337.7 [Processing: rae_camera, rae_hw]

                337.7 [Processing: rae_camera, rae_hw]

                337.7 [Processing: rae_camera, rae_hw]

                337.7 [Processing: rae_camera, rae_hw]

                337.7 Aborted <<< rae_hw [5min 33s]

                337.9

                337.9 Summary: 3 packages finished [5min 37s]

                337.9 1 package failed: rae_camera

                337.9 1 package aborted: rae_hw

                337.9 2 packages had stderr output: rae_camera rae_hw

                ------

                Dockerfile:19

                --------------------

                17 | RUN cd .$WS/ && apt update && rosdep update && rosdep install --from-paths src --ignore-src -y --skip-keys depthai --skip-keys depthai_bridge --skip-keys depthai_ros_driver --skip-keys audio_msgs --skip-keys laserscan_kinect --skip-keys ira_laser_tools

                18 |

                19 | >>> RUN cd .$WS/ && . /opt/ros/${ROS_DISTRO}/setup.sh && . $UNDERLAY_WS/install/setup.sh && colcon build --symlink-install --cmake-args -DCMAKE_BUILD_TYPE=${BUILD_TYPE}

                20 |

                21 | RUN echo "if [ -f ${WS}/install/setup.zsh ]; then source ${WS}/install/setup.zsh; fi" >> $HOME/.zshrc

                --------------------

                ERROR: failed to solve: process "/bin/sh -c cd .$WS/ && . /opt/ros/${ROS_DISTRO}/setup.sh && . $UNDERLAY_WS/install/setup.sh && colcon build --symlink-install --cmake-args -DCMAKE_BUILD_TYPE=${BUILD_TYPE}" did not complete successfully: exit code: 2

                    garymok You didn't even post the command youre trying to use to build the Dockerfile- the project builds just fine for me.

                    They may have merged changes since then, however

                      @garymok Could you post an issue of this error on the repo itself? This would get you in contact with ros dev team directly.

                      garymok apologies, they merged some PR's since and it's currently broken. Their repo should be ignored for now, I think. Pack away the rae, it's no use atm.

                        garymok Check the repo again, just use their prebuilt image on docker hub

                          Just received my rae. Trying to get it on to the internet. Error at the second step is:

                          root@keembay:/etc# wpa_supplicant -B -i wlp1s0 -c /etc/wpa_supplicant.conf

                          Successfully initialized wpa_supplicant

                          nl80211: kernel reports: Match already configured

                          nl80211: Could not configure driver mode

                          nl80211: deinit ifname=wlp1s0 disabled_11b_rates=0

                          wlp1s0: Failed to initialize driver interface

                          reset rae and tried again. No luck. It is not able to join the network and get a DHCP address. What could I try next?

                          Also noticed that after the first login via wifi, I have only been able to login via usb.

                          Tx.

                          • erik replied to this.

                            Hi Wijay ,
                            Could you try following Robothub installation? That will also configure the wifi before connecting to the robothub cloud.

                                hi @Wijay, I found that I needed to enter the command wpa_supplicant -B -i wlp1s0 -c /etc/wpa_supplicant.conf twice (fails the first time).

                                The following sequence of commands works for my rae …
                                systemctl stop systemd-networkd
                                wpa_supplicant -B -i wlp1s0 -c /etc/wpa_supplicant.conf
                                ps ax | grep wpa_supplicant # If process isn't running, try again
                                wpa_supplicant -B -i wlp1s0 -c /etc/wpa_supplicant.conf
                                systemctl status systemd-networkd # Network daemon should have restarted automatically
                                ifconfig wlp1s0 # Interface should be up and have an IP address

                                  hi @erik, whilst the Luxonis RobotHub looks useful … it is a good thing for the rae (and all other products) to be able to operate independently of RobotHub in other cases. After all, it is running Linux … and Wi-Fi should just work 🙂

                                  I haven't figured out yet why the wpa_supplicant process is failing on my rae … the first time that the wpa_supplicant -B -i wlp1s0 -c /etc/wpa_supplicant.conf command is entered and then succeeds the second time, but will diagnose further.

                                  Regarding using the Luxonis RobotHub to configure Wi-Fi on the rae robot … entering one's personal Wi-Fi network SSID and password on a third-party web-site is a security risk, which some people may be comfortable with, but really is a poor security practice that should be avoided. At the very least, offering an open-source tool that can be reviewed and run locally to produce a QR-code on the desktop would be better … thanks !

                                  Note: I realize that completing a KickStarter hardware project … including all the software, firmware and documentation is a huge endeavor. Please don't take my comments as being overly critical … I like the Luxonis products, as well as the commitment to open-source.

                                    erik Hi Erik, I assume you are referring to the method where you scan the QR code? Tried that first and it does not work.

                                      geekscape Ji geekscape

                                      Thanks. Running the wpa_supplicant -B -i wlp1s0 -c /etc/wpa_supplicant.conf command the second time works.

                                      Many thanks. Let me proceed and see how I go.

                                          Stupid question. Where is the remote control app for rae? Or is it served as an http page on rae itself?

                                          Thanks.

                                            Wijay

                                            Sorry. Figured out that it is <ip address of rae>:9010.

                                              geekscape @erik

                                              Something that could be added to docs would be to use wpa_passphrase <ssid> to generate part of the wpa_supplicant.conf file and not have the password in plaintext anywhere by not copying the #psk= portion of the output.