• DepthAI

  • Why I can not get 120 fps while USB3.0 only use 23MB/sec traffic?

Hi, I am using OAK-D to analyze golf ball's flight path. So 120 fps of images is very important. I am testing using:
python3 depthai_demo.py -cam left -monor 400 -monof 120 -dnn -dd
I get about 70 to 90 fps only using a I5 CPU with USB3.0 hub. I have monitored the USB traffic on OAK-D, and it's about 22 ~ 23 MB/sec only. So it seemed it's not the USB3.0's 5Gb thal lower the fps. And I have disabled depth and AI by -dd,-dnn.
Do I need a I7 faster PC to get 120 fps? But why? I have turn off depth and AI, it seems I do not need to do much calculation to use a I7 CPU.
Or there's any switch to set to get 120 fps?
Currently I need raw images only, is the firmware on OAK-D slow down fps?

Hello @frankshieh ,
so the reason behind this issue is the XLink - protocol that we use between host/device. With the new version of depthai 2.10.0.0, we have increased XLink packet size which will improve throughput by quite a bit - I am able to get 117.3 FPS running the code above. We haven't updated depthai_demo to 2.10.0.0 but will do today (you can do it yourself by changing depthai==2.10.0.0 in requirements.txt)
Thanks, Erik

    erik Hi, I have tested new depthai==2.10.0.0. Yes it helps , but not a lot. I can run to about from 80 fps to over 90 fps. If I add -usb usb2 to lower USB data rate then I can get 103 fps. The data rate over USB when using MJPEG compression reduced from about 23MB to 14MB. But I still can not get nearing 120 fps.
    The previous stereo camera we used, I can get 200 fps easily using a I3 computer just as it has USB3. The call back function we hook on to get every frame and won't miss any frame just as we do not install too many background running apps. We were using C++ and running the call back function as high priority thread so we almost did not lost any frames.
    This is important for us NOT to lost any frames. As we are analyzing golf ball flying with over 150 mph speed, the ball images in a swing catched might as low as 3 ~ 5 images. Lost any might let us failed to analyze ball's speed and direction. I do not know what's the bottlenecks that keep OAK-D from getting 120fps when running depthai_demo.py?

    I think your computer must be a high end PC, so I guess if I get a I7 will still lost some frames. Do you think it's caused by Python being interpreter and using C++ will help?
    Please advice, thanks!


    • erik replied to this.

        Hello frankshieh,
        I don't think it's possible that frames would drop - I believe the only limitation is the throughput of the XLink. We are adding support for USB 3.2 10GBPS (superspeed+) so if you have a host that supports that, it would definitely help. My host isn't anything special, but it does support USB3.2 (superspeed).
        From my observations your host/USB cable doesn't support USB3 so you get the same FPS with USB2 mode; could you try running rgb_preview.py (it will print the USB speed to the terminal)?
        Using USB3 should solve your issues.
        Thanks, Erik


            I am using USB3.0. I will buy a USB3.2 (superspeed) pci add on card to try and post the result.

            This is the result running:

            python3 .\depthai_demo.py -s left right -monor 400 -monof 120 -dnn -dd
            It's still using 44MB/sec (using HDD Device Monitor Studio 8.36) which is still far less than typical USB3.0 transfer maximum speed 320MB/sec.
            It seems OAK-D has not yet got too heavy loading on USB3.0 bus, but I will try USB3.2 to see the result.
            We are replacing previous stereo cameras on our existing systems with OAK-Ds. And these existing PC we used Intel NUC which has no PCI socket to add and only with USB3.0 ports. If only with USB3.2 to get near 120fps, that means we have to use new NUC with USB3.2 to replace old USB3.0 NUCs.
            So, that why I am checking can USB3.0 to get 120fps or not. And we do appreciate Luxonis can help us on this!

            Oh also IIRC 118FPS is currently the max framerate on the global shutter grayscale right now. And also there may be some error in counting the framerate as well that could be impacting here.

            I got abut 110 fps using I5+USB3.0. I think this will be OK for my application just as the lost images are not continueus. And if I set it to -monof 100, there will still lost frames and the fps only applear as 96 fps. It might be fps counting problem.

            Hello frankshieh ,
            C++ doesn't have depthai_demo alternative, so you would have to construct your pipeline yourself with our API. You would use pipeline.setXLinkChunkSize(chunk_size) setter to set the XLink chunk size.
            Thanks, Erik

                erik Hi, I got a USB3.1 Gen2 card added. The speed is 117.3 fps running :
                python3 depthai_demo.py -s left right -monor 400 -monof 120 -dnn -dd --xlink_chunk_size 0.
                But rgb_preview.py still print USB speed as SUPER, it seems rgb_previwer.py does not idtetify SUPER HIGH(10Gb) from SUPER(5Gb).

                • erik replied to this.

                    Hello frankshieh,
                    the 10GB feature is still in development, should be out in main in 1/2 weeks🙂 You can try it with the branch develop_embedded (depthai-python repo).

                      a year later

                      GergelySzabolcs

                      Using ">python3 depthai_demo.py -s left right -monor 400 -monof 118 -dnn -dd -xls 0" I'm only able to get 50FPS. Windows + Xeon 6 core CPU with nVidia GPU.

                      C:\depthai>python3 depthai_demo.py -s left right -monor 400 -monof 118 -dnn -dd -xls 0
                      Using depthai module from: C:\Users\live\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.8_qbz5n2kfra8p0\LocalCache\local-packages\Python38\site-packages\depthai.cp38-win_amd64.pyd
                      Depthai version installed: 2.17.1.0
                      Setting up demo...
                      Available devices:
                      [0] 19443010B18DD71200 [X_LINK_UNBOOTED]
                      USB Connection speed: UsbSpeed.SUPER
                      No depth-related previews chosen, disabling depth...
                      [2022-07-22 22:25:53.170] [warning] Insufficient permissions to communicate with X_LINK_BOOTED device having name "2.16". Make sure udev rules are set
                      Stopping demo...
                      === TOTAL FPS ===

                      • erik replied to this.