• DepthAI-v2

  • [Android] OAK-1 and OAK-D Low frame rate

Hi !
I'm actually working on a project where I need to plug an OAK-1 camera to an Android device.
I took inspiration from this projet : https://github.com/ibaiGorordo/depthai-android-jni-example

My goal is to get the YUV420 directly from the ISP in 4K at a frame rate of at least 20FPS (using the C++ API), I saw on the documentation that these numbers were normally largely attainable.

But my problem is that when I connect the OAK-1 to an Android device (Note 9 and others...) and try to get the image from the ISP the frame rate is only 4 FPS, even with a SUPER usb speed.
In fact the same code run around 28FPS on my Windows machine.

I wanted to know if this problem was known and if someone managed to overcome it ?
Or if someone has an idea of where to explore on the side of depthai-core or its dependencies to try to speed up this frame rate.

Any help would be appreciated 😀

  • erik replied to this.

    Hello Brac ,
    I believe that is because two things:

    • Our Android implementation (depthai-android-jni-example) isn't the most performant implementation in regards of copying some messages around, so there's the first place of improvement
    • 4k ISP frames is A LOT. Eg. RPi 4B can't even handle 20FPS 1080P frames, just to give you an idea of low fps due on low-end computers (CPU). I would suggest lowering resolution and/or encoding (example here) the stream (mjpeg/h264/h265) to reduce bandwidth needed.

    Thoughts?
    Thanks, Erik

      Hello @erik ,
      Thanks for this quick answer ! and sorry for this late return...

      For the first point I known that this example was not the most performant, I just linked it to put some context on what I was doing.
      In fact, I started from this example by removing everything that could have "side effects" on performance and just keeping a callback on the ISP output in the C++ code.

      Indeed 4K is a lot, but the objective was to have the best possible resolution without loss of data during the transfer. That's why the resolution and encoding was not necessarily the first choice ^^

      By lowering the resolution to 1080P and by using the video encoder (MJEPG) I can reach about 15 to 20FPS, but a the expense of a loss of precision on the output of the detection algorithm that will consume the image.

      Thank you for taking the time to answer my question 😃

      I'll let you know if I find an effective implementation to this problme 👍️

        Hi Brac ,
        Interesting, thanks for the feedback! I would have thought android phones were more performant and there wouldn't be an issue. Are you also displaying/decoding the frames (15/20FPS) or just reading them and discarding them?

          6 days later

          Now I use the 1080P video pipeline encoded in MJPEG.

          I create a cv::Mat from the Java side and send the native address to the native side where I fill the Mat.
          (it seems more efficient than passing an array of integers as in the example).

          If I display the video feedback using a Bitmap (like in the example) I get around 14 to 18FPS
          And if I display it through OpenGL (ES2.0) I get around 18 to 22FPS which is starting to be intresting ^^

          I am still surprised because I use a device with a Qualcomn XR1 and I would have expected more performance.

          Work in progress ^^

            erik changed the title to [Android] OAK-1 and OAK-D Low frame rate .

            Hi Brac ,
            This is looking really promising!! If your code is opensource we would love to link it from our depthai-android example repo. Otherwise I can also just mention the details (eg OpenGL for faster live stream) in our repo.
            Thanks, Erik

              3 years later

              • nik

                • Post #8

                Hi @Brac, I am currently working on the same problem with following setup.

                Luxonis Oak-1 USB + Pixel 8a

                Could you please opensource your fix for getting stream from camera to Android?