• DepthAI

  • How to increase OAKD-lite's stereo pair frame rate?

Thereotical OAKD-lite's stearo pair maximun frame rate is 200fps. But the actual tested result is about 80100 fps, is there any tuning method to increase the frame rate? I am using stereo images only and currently for capturing raw images!

  • erik replied to this.

    Hello frankshieh ,
    Could you share the full pipeline/code you are using? I believe depth could reach 200FPS without any special modes postprocessing filters enabled, at 400P resolution. Thoughts?
    Thanks, Erik

      https://github.com/frankshieh/depthai-python/blob/main/mono_preview.py
      https://github.com/frankshieh/depthai-core/blob/main/mono_preview/mono_preview.cpp
      This is the codes I am testing for fps! The python gets about 102 fps while the cpp version gets about 94 fps.
      And I am sure the fps is not slowed down by my PC's USB3 bus, I can get a 200 fps 640x400x2 stereo video stream using the same PC for another 3D camera.

      • erik replied to this.

        Hello frankshieh ,
        I think the main way to improve fps is to set pipeline.setXLinkChunkSize(0)
        I have tried with OAK-D-Lite and I get about 115FPS on each mono camera with this script. Mono image sensor (OV7251) currently supports max 117FPS at 400P. Could you try it out?
        THnaks, Erik

            Hi, I tried and I can get 113 fps using your script but this is still far behind 200 fps. If I want to add a 200P cropped image mode with your source codes to increase the fps, where shall I start? Or the 120fps is limited by internal pipeline and nothing I can try to increase it for current internal firmware?

            • erik replied to this.

              Hi frankshieh , currently that's the limit by the sensor config. I believe the current bottleneck is ISP speed (3A algorithms) - as that's combined 240 FPS. I believe we plan to update the ISP so 3A algos will only run once every eg. second or third frame, so this won't be bottleneck anymore and we can achieve higher framerates of these mono cameras (OV7251).
              Thanks, Erik

                Hi, when running high frame rate, we usually do not need the ISP 3A. The gain and shutter speed are usually fixed. So maybe you can have an option to disable ISP for users like us. We need higher fps and we do NOT need ISP 3A, and ISP 3A will surely disturb our application. When we need higher fps, we need extra LED lights in a short period. And we would like to set the sensor's gain and shutter speed our own manually and NOT by the 3A algorithm. Because these extra LED lights are not always on due to the high power consumption, and we only turn LED lights on when we detect the object is passing and falls in our view area of camera. And the 3A algorithm must have to be disable in our application. Otherwise the gain and shutter speed calculated by 3A will make the images to bright when we turn the LED light on for a short period(maybe onlt 60 frames@200fps)

                2 years later
                8 months later

                Hi @erik,
                Is there any update on this?
                I just bought this camera because I have a project that needs the 200 fps. The product info for this camera says the mono's can do 200 fps (Luxonis Oak-D-Lite - KI-Bilderkennungskit (botland.de)).
                But when I run it (on a raspberry pi 4), using the (mono_previw.py script )[luxonis/depthai-pythonblob/1a1eb16e94874e88e14594423e34cb3d69828a6c/examples/MonoCamera/mono\*preview.py], I get a message """MonoCamera OV7251: capping FPS for selected resolution to 99"""
                I have tried changing the resolution to THE_480_P, and get the same.
                I guess the reason for the lower frame rate is the live post-processing. Does the 200 fps only mean that the RAW image data is produced at that rate?
                I there any way to bypass the post-processing to reach that 200? In my case, live viewing is not important. It is fine to record as fast as possible, then process the frames some time later.

                  Hi MarkyG
                  The specs are wrong. Max FPS for OV7251 is 120FPS (117 FPS at 400p).
                  Stereo should work at about the same FPS but you won't be able to view frames that fast.

                  You would also need to utilize the full USBSPEED_SUPER (10Gbps) bandwidth to send frames back to the device without dropping them.

                  Thanks,
                  Jaka

                    Hi @jakaskerl,

                    Bummer that it can't do 200fps.

                    Regarding the USBSPEED_SUPER, is that a configuration option that is accessible in the python library?
                    I don't think the raspberry pi 4's USB 3 can do 10Gbps.

                    Thank,

                    Mark

                      MarkyG
                      USBSPEED_SPEED is accessible in python but it's used to limit the speed of USB (SUPER_PLUS == 10gbps, SUPER==5gbps - mistake in previous post...).
                      I think RPI usb is 5gbps, yes.

                      You just need to send smaller amount of frames to host side (or just send the results). It should still work, though not at 200FPS.

                      Thanks,
                      Jaka