• HardwareROS

  • Is an OAK camera appropriate for my usecase?

Hi,

I'm wondering if an OAK camera is appropriate for my usecase & is the best option, or if there might be better options. And if an OAK camera is appropriate for what I want to, then which cameras would be best.

I need both a depth camera as well as 4 camera for a rover-like platform.
More specifically, I'm part of a university team participating in URC and CIRC.

For the depth cameras, I have the following requirements:

  • Can be used without too much difficulty on an NVIDIA Jetson Nano (Specifically, at the moment we're using the dev board. I believe we're using the Jetson Orin Nano, though I'm not 100% sure on that)
  • First-party integration with ROS or rather easy to integrate with it.
  • Being able to get a mesh or point cloud would be a major bonus.
  • We're currently using a USB connection, however I would be open to something like GMSL2 if options for how to connect it to a jetson nano dev board could be provided.
  • From my understanding of the benefits/downsides of different technologies for a depth camera, I believe that we want something that uses a stereo system, however a combination of stereo + IR would be a bonus.
  • Budget is roughly $600 USD (we were previously using a ZED 2 camera however it broke and we are considering new options) (can possibly be extended upwards slightly if a really good option is presented)

For the 4 other cameras, I'd ideally like to use the same model of camera for all 4. The cameras should ideally be at most \$100, preferably closer to \$80.
Here are some requirements I have for those:

  • USB-based connection would be best. If there is a camera which isn't usb-based but offers significant benefits or price savings, then I would be open to using something else.
  • an enclosure is not necessary, as we can 3d print our own enclosures.
  • small footprint/form factor. We're currently using arducams but are looking to replace them. (I believe we're using the EVK modules? Although my understanding is we're using prototypes of it that were donated to us a few years back, so they may be different from that)

Here are some requirements that apply to everything:

  • 1080p is a minimum for the resolution. Higher is better. Higher quality for the depth camera has greater priority over a higher resolution for the other cameras.
  • 30fps is a minimum for the frame rate. Higher resolution would be preferred over higher frame rate, if a choice between the two is necessary. (ie. 4k@30fps > 1080p@60fps)
  • on-camera hardware encoding (h264, optionally h265 or av1) is desired (ideally at a higher quality than what the jetson nano can offer with its hardware encoder), although can be compromised on if absolutely necessary.
  • Cameras need to be able to perform reasonably well in a wide range of environments. It may be used both in the day in the desert where there will be a lot of light reflection from the sand, as well as in very low-light conditions at midnight. (We may be using a light, however having the ability to perform well without the light would be a very nice bonus)
    ideally we'd be able to perform in both environments without any hardware changes, however adding something like a light filter would be acceptable.
  • I would really like the ability to do opencv processing on the cameras. More specifically, detection of aruco tags, although possibly some additional stuff (like general object detection) in the future. It doesn't need to actually be opencv based, as long as we can do aruco tag detection on it and porting the aruco tag detection code isn't too difficult.
    My reason for wanting this is because currently we're doing aruco tag detection for the camera feeds, however in order to do this we are running it through a ROS node reading from an image topic published by the usb_camera node. This then interferes with what we're using to stream the camera feeds wirelessly using gstreamer, because only one process can read from a block device in linux. So instead we need to have gstreamer read from the image topic, which seems to heavily limit the frame rate to something like 10fps even if I attempt to increase it. Because of this we're currently limiting it to only doing aruco tag detection on a single camera and just taking that frame rate hit for that one camera, but if we could run it for all the cameras instead that would be amazing & doing it on-camera would simplify things quite a bit, I feel.

I plan to use these cameras with ROS for autonomous navigation using nav2 (this will be performed on-rover), as well as for streaming them over a wireless connection.

Also, if you have a lot of experience with other general hardware things, there are a few other components like GPS modules or an IMU that I am looking for suggestions for. I won't go into it here as it's off-topic, however if you'd like to provide suggestions you could look at my fedi post I made about it. If you don't have a fedi account then it's probably (hopefully) fine to post it here.

Hi,

Regarding the depth camera requirements, OAK-D Pro is your best choice (or possibly OAK-D Pro Wide):

For the 4 extra cameras:

You can go with 4x cheap USB UVC board cameras (like Arducam IMX323/IMX291, 1080p30, H.264).

But you may also want to look at our OAK-FFC-4P: https://shop.luxonis.com/products/oak-ffc-4p

It lets you connect 4 sensors (mono or color) to one DepthAI module with onboard compute. You can run models across all 4 cameras, cleaner cabling, and less load on the Jetson.

Best,

Lovro

    lovro Built-in H.264/H.265 encoder

    What is the quality of the on-board h264 & h265 encoders like? Is it using hardware-accelerated encoding or software encoding? And would you by any chance know how it might compare to the hardware-accelerated encoding on the Jetson Orin Nano?

    lovro You can go with 4x cheap USB UVC board cameras (like Arducam IMX323/IMX291, 1080p30, H.264).

    we were previously using arducams, but I'm quite dissatisfied with their quality. not only is the resolution on the ones we have quite bad, but it also seems to be cranking the hell out of the saturation on them to the point where I need to decrease it by like 75% with gstreamer, massively degrading the quality of the video feed.

    the OAK-FFC-4P does look interesting, however the cameras will have some significant distance between them, up to ~1.5m-2m so I don't think running a ribbon cable over that distance is the best idea
    is there anything which is able to take some cable connection that can be bundled up into a single wire? Doesn't have to be USB-A or USB-C, could be something like GMSL2 or whatever since it's just going to that board.

    also, it says that the OAK-FFC-4P uses RVC2, which from what I read on the wiki is not able to do on-board processing of AprilTags, which I would assume is quite similar in complexity to ArUco tags.

        2 months later

        hey, it's been a while since I replied, so I'm going to bump this.

        the OAK FFC 4p also notes that it's limited to 2 4-lane MIPI connections and 2 2-lane MIPI connections. how exactly does this translate into resolution?

        I did also notice that there's the OAK4 SoM, which uses RVC4 instead of RVC2. However, there is no information that I can find on how you'd integrate it with a camera or any other system.

        it doesn't seem like there's any FFC system that uses RVC4.

          5 days later

          Hi solonovamax
          4 lane is generally only used for 12MP or high framerate 4K. You should be fine even on 2lane for most configurations.

          We don't have FFC-style RVC4 devices yet apart from the devkit, but they are in the works.

          Thanks,
          Jaka

              5 days later

              jakaskerl We don't have FFC-style RVC4 devices yet apart from the devkit, but they are in the works.

              any idea on how long it might take?

              also, what is the quality like on those the h264 & h265 encoder? and would it be able to handle 4x 1080p 30fps streams while also doing something like SLAM or YOLO?

                  solonovamax
                  The more demand we have, the more we can prioritize this. I would say not before 2026 - we are currently working on the EE design.

                  solonovamax also, what is the quality like on those the h264 & h265 encoder? and would it be able to handle 4x 1080p 30fps streams while also doing something like SLAM or YOLO?

                  I would say yes, though we still have to test it. From the specsheet - "4K @ 120FPS encoding for H264 and H265".

                  Thanks,
                  Jaka