How to install rae_sdk

IvoGermann

With firmware 1.14 the python sdk is available. According https://docs-beta.luxonis.com/develop/rae/rae-sdk to use it requires the right firmware and the right agent number. I've checked that on my RAE and all is ok. But trying to use it in the RaeStarterApp it turns out that raesdk is not installed. When I try to install it with 'pip3 install raesdk' I get: 'ERROR: Could not find a version that satisfies the requirement …'
What is the right way to install the rae_sdk?

jakaskerl

Hi IvoGermann
Afaik, updating to the correct Luxonis OS version and agent version, then setting the latest image should be sufficient to run the Rae SDK.

Thanks,
Jaka

IvoGermann

Hi Jaka,
Thanks. I'm a step further but still think, I miss something. I'll explain what I did:
- Started RAE and started RobotHub
- In Overview I checked the FW version = 1.14 and Agent Version = 23.223.1855 -> all ok
- Assigned RAE to the StarterApp and opened the file robotapp.toml
- In section [runtime.runs_on] I changed name = "luxonis/rae-ros-robot:dai_ros_py_base" to name = "ghcr.io/luxonis/rae-ros:v0.2.2-humble"
- Then I pressed the button to apply this change which took quite a while for the app to restart.
- Then I started a console, opened python3 and tried 'import rae_*sdk -> works fine
- Ok, so far so good. Then I tried '*from rae_sdk import Robot' but got the error: ImportError: cannot import name 'Robot' from 'rae_sdk' (/ws/build/rae_sdk/rae*sdk/init.py)
How can I get the Robot import running?

=> Just an idea: I think when importing rae_sdk it points to the wrong init.py. The correct path should be /ws/build/raesdk/raesdk/robot/init.py. But this is really only a wild guess…*

DaniloPejovic

Hey,

you need to import following way:

           `from rae_sdk.robot import Robot`

not directly from rae_sdk.

IvoGermann

Hey Jaka,
You made my day - thanks a lot! I think it is important to note that the RaeStarter app has to be started first. Then using another console or running python scripts works fine. Really a great step.
Maybe the examples in https://docs-beta.luxonis.com/develop/rae/rae-sdk should be updated accordingly.

IvoGermann

Hi Luxonis team,
The page https://docs-beta.luxonis.com/develop/rae/rae-sdk doesn't work anymore. Did you place it to somewhere else?
=> Sorry, just found it in https://docs-beta.luxonis.com/hardware/rae/rae-sdk/ => all good.

gregflurry

@IvoGermann I am trying to achieve what you appear to have achieved. despite being a software developer for 40 years, I don't apparently have enough skills/context/something to do so. I apologize upfront, but I hope you can help me. If not, no worries.

What is the RaeStarterApp? Where did you find it? How did you install it on the device?

Where did you find robotapp.toml? I don't see one on my device.

When you say "I pressed the button to apply this change", what button do you mean? Maybe the Restart in the Hub?

When you say "opened python3 and tried 'import rae_*sdk -> works fine" does that mean you ssh'ed into your device and issued the command python3?

You say "it is important to note that the RaeStarter app has to be started first". Do you think that any app can be running? So far I've only installed and a couple of pre-built apps using the Hub. In these cases, it seems that only one app can be using the device at a time, but I confess I've not tested that assertion.

Any other advice you can give me would greatly help. I've become very frustrated trying to do anything with my rae.

Thanks.

IvoGermann

gregflurry
Right, it has also taken me quite some time to get the RAE working in a way. Still, I haven't been able to program anything that comes close what the device should be capable off (i.e. visual SLAM). However, I can explain how I use it with the Python/ROS2 interface. I hope that helps a bit.

Make sure you have installed the firmware Luxonis OS 1.14 which is required for the RAE SDK. See here https://docs.luxonis.com/hardware/rae/rae-sdk/
Go to Robothub https://robothub.luxonis.com and make sure your RAE appears there. If you don't see your device then you have most probably not configured properly the WIFI. Hint: Dont' use any special characters in the WIFI password as RAE has poblems with it.
It shoould looks like this:
When you click on your device then go to 'Perception Apps' and install the 'RaeStarterApp'. It should appear here:
Go to the RaeStarterApp and cllck on 'Reassign OAKs' and select your device:
Start Code Editor and if you do the first time you can configure the view. In my case iit looks like this:
Ok, the fun can start. Open the terminal and copy the following lines into it:
source "/opt/ros/$ROS_DISTRO/setup.bash"
source "/ws/install/setup.bash"
export ROS_DOMAIN_ID=30 # setup domain id
Start the robot with: ros2 launch rae_hw control.launch.py enable_battery_status:=false
(Check for the detailed ROS2 commands https://docs.luxonis.com/hardware/rae/rae-sdk/)
The good thing with ROS2 is that your RAE is accessible from any other Ubuntu ROS2 computer in your subnet now. I use either a VM with Ubuntu 22.04 and ROS2 installed or my native Ubuntu installation. Just open an editor, make sure it is bashed and go to the same domain with: export ROS_DOMAIN_ID=30
However you are ready to check on your RAE with i.e. ros2 topic list:
Or you can move your RAE with keys by:
ros2 run teleop_twist_keyboard teleop_twist_keyboard
then you can move the robot with keys:

Ok, that's it. Maybe one last hint: There is no chance to use the RAE without getting used to work with ROS2. Make yourself familiar with it then the little flat brick comes into life. In my case I made a Udemy course.
And the very last comment: No I do not use docker as I still don't really understand how to get the RAE running with it. I hope someone comes up with a clear step-by-step guide how to do that.
Good luck!

gregflurry

IvoGermann Thank you so very much for your help!

I confirmed that my device is running the agent 23.223.1855, per steps 1 and 2. I added a screenshot of what I think is the equivalent of your screenshot for step 2. I am apparently using an updated Hub, and it looks different. First, I don't see the error message, and second, my device is named 'rae' rather than 'keembay', though I may have done something 6 months ago to change that. .

When I click on the device I get a a very different view, as shown in the screenshot below. There is no tab called "Perception Apps"!

Using the URL for the legacy Hub in step 2, I get something quite similar to your 3 screenshot. However, the only app I see is one I downloaded two days ago. There is no RaeStarterApp pre-installed.

When I click on Install App I get the following popup.

When I click Run I see the following popup. Again, no RaeStarterApp.

So, once again, not sure what is going on, nor is there documentation enough to begin to understand the situation.

In any case, I went back to the basic Perception Apps tab and clicked the Car App. The following screenshot shows the results. I can find nothing that lets me open Studio to view/edit the code! Just to make sure I'd not missed something, I tried the same thing in the new Hub, and again, no way that I can see to open Studio.

Once again, I'm stymied. That said, again, I very much appreciate your help.

gregflurry

IvoGermann IvoGermann I've continued to work on this in between other projects. I've made some progress thanks to you and @MartinMotycka. I now have I think achieved step 5 in your list; however, I've done it with the new Hub version, which seems to allow me to install/develop a somewhat different set of code that is called "RAE Starter"; here is what I see after installing it (I gave it the name 'Maybe RAE Starter'), assigning rae to it, opening up Studio, then starting it, and poking around in the source:

There are a couple of things worth noting. The code is different; for example, I see no images in the source. I note that the start_ros.sh contains the code in your step 6 and your step 7.

At this point I am still quite confused. The rae-sdk page mentions "ghcr.io/luxonis/rae-ros:v0.2.2-humble" as needed in a "robotapp.toml". It is not in that file for the app (see the right side) of the screenshot). Further, in steps 8 and 9 you are using ros2 commands to run stuff. I know next to nothing about ros(2), but I'd expect to run the rae Python SDK using a python3 command.

Any elaboration would be much appreciated.

IvoGermann

gregflurry Ok, that looks very different and beside the penguin is wrtten 'rae'. in my case it is written 'keembay'. Sorry, but I still think you are on OS 1.12 which is the OS that was initially also on my RAE. Then I had to update it to 1.14. Did you update to 1.14?
Unfortunately with the newest RobotHub version the OS version is not in the Overview anymore. But the old version still works: https://robothub.luxonis.com/ - Can you check the version in the overview? Maybe the update didn't work.

MartinMotycka

Hello gregflurry

To run Code Editor the app has to be installed in the develop mode.

Go to apps --> create new app. You can install it in the develop mode from the app overview. Second possibility is to go to "install app" and the install it from there in the develop mode. When installed go to device --> app --> app detail and there you will find the Code Editor

Note: We do not recommend to use the legacy front end. It might not work as expected.

jakaskerl

cc @MartinMotycka

gregflurry

IvoGermann I started the old Hub version, and it reports that I am running OS 1.14. The mystery continues.

gregflurry

MartinMotycka I have in the past created a new app using the <> Develop option. That did give me access to the code editor. It did me little good due to the little documentation on what the code means.

Using the latest Hub version, I installed an app via the Run option. I could find no magic incantation that allowed me to indicate I wanted to develop it, use the code editor, or anything of the sort either during or after installation. Do you have any example screen shots?

Further, I really want to develop using the new Python SDK. I can find nothing that shows how to do that. I think there are example apps on GitHub, but I have no idea how to get them onto the rae, run them, or leverage the web UI that some of them seem to include.

Relative to a UI, I have the Car App installed. I cannot find any way to use it. There must be some UI somewhere, likely a web based. Do you have any hints about where to look for the UI?

Thanks for the advice.

MartinMotycka

Hello @gregflurry .

It is not possibile to develop app that has been installed in the run mode. In develop mode the app runs completely on the device and you need direct access to the device via local network.

To run the local frontend, go to devices -> select your device from the list and go to Apps tab:

And there you find the local frontned:

If you install the app in develop mode, on the same place is the code editor.

gregflurry

MartinMotycka Thanks. Your description has helped me make progress!

First, let me describe my experiences with Run installations. I already had the Car App installed and rae assigned to it. I started it and then clicked Local Frontend. I got a warning Safari message about potentially malicious website. I pushed through to open the page and got a "frontend" with some controls. I was able to move the rae and the LEDs and the speaker. I was even able to get it to "follow me"! [as an amusing aside, while showing my wife, rae started to follow her :-)]. Progress! That said, apparently the wifi connection was poor or something, as the frontend lost connection a few times.

I then installed in Run mode the Default App. It too ran, but apparently the same front end. This in retrospect makes sense, but I realize now that I'd seen the front end before, I just did not realize it was associated with multiple apps. Oh well.

Second, I next tried to install a Develop app. I was successful in installing a Streaming template, getting it started, and viewing the video streams. I also opened up the code editor. Great! All thanks to your guidance. That said, I now remember I was at this point back in March, but activities seem to be a little bit different, and the overall system seems a bit faster and a bit more stable.

That said, I think the code I'm looking at is based on ROS2. I really want to build apps on the Python SDK. I seem to be no closer to that. Can you give me any guidance on how to do that?