MartnTous

Hi @jakaskerl

I assumed it had to do with hardware limitations, but after running in debug mode, I noticed that the memory and cpu usage is low, which I found strange (this can be seen in the image I attached above).

As for the ISP processing capability being 600 MP/s I also find it odd that this is a limiter, since I am working with ISP downscaling. Even if I wasn't using downscaling and working with the 12 MP, I'm working at 5fps, with two cameras at a time, so 12x5x2=120 MP/s, which is considerably less than the 600 MP/s capacity.

Maybe it's another limiting hardware component? Or does it have to do with my acquisition script (maybe there is a way to make it more efficient)?

Hello everybody!

I am using two IMX378 cameras connected to an OAK-FFC 4P module. I need to improve the delay between image acquisition and display on screen, which is more than one second. At the same time, I have a very low fluency in the video, this is because it does not work at the fps that I indicate. Below is an image that shows the delay mentioned and the fps.

I don't know if it is a hardware limitation and therefore a delay that I have to deal with, or if there is any possible way or technique to improve this code.

If I do not use "cam1.setIspScale(1,2)" the delay is ~4 seconds.

import cv2
import depthai as dai
import time

def getFrame(queue):
    frame = queue.get()
    return frame.getCvFrame()

if __name__ == '__main__':

    fps = 10
    frame_count = 0
    
    # Define a pipeline
    pipeline = dai.Pipeline()
    device = dai.Device()

    # DEFINE SOURCES AND OUTPUTS
    cam1 = pipeline.create(dai.node.ColorCamera)
    cam2 = pipeline.create(dai.node.ColorCamera)
    cam1.setBoardSocket(dai.CameraBoardSocket.CAM_A)  # 4-lane MIPI  IMX378
    cam2.setBoardSocket(dai.CameraBoardSocket.CAM_D)  # 4-lane MIPI  IMX378
    cam1.setResolution(dai.ColorCameraProperties.SensorResolution.THE_12_MP)
    cam2.setResolution(dai.ColorCameraProperties.SensorResolution.THE_12_MP)
    cam1.setIspScale(1,2)
    cam2.setIspScale(1,2)
    cam1.setFps(fps)
    cam2.setFps(fps)
    # Set output Xlink 
    outcam1 = pipeline.create(dai.node.XLinkOut)
    outcam2 = pipeline.create(dai.node.XLinkOut)
    outcam1.setStreamName("cam1")
    outcam2.setStreamName("cam2")
    # LINKING
    cam1.isp.link(outcam1.input)
    cam2.isp.link(outcam2.input)

    outcam1.input.setBlocking(False)
    outcam2.input.setBlocking(False)
    outcam1.input.setQueueSize(1)
    outcam2.input.setQueueSize(1)

    with device:
        device.startPipeline(pipeline)
        cam1 = device.getOutputQueue(name="cam1", maxSize=1, blocking=False)
        cam2 = device.getOutputQueue(name="cam2", maxSize=1, blocking=False)
        n=0

        while True:
            frame_count += 1
            n+=1
            
            elapsed_time = time.time() - start_time
            if elapsed_time > 1.0:
                fps = frame_count / elapsed_time
                print(f"FPS: {fps:.2f}\n")
                frame_count = 0
                start_time = time.time()

            start_adq = time.time()  # Inicia el temporizador

            Frame1 = getFrame(cam1)
            #Frame2 = getFrame(cam2)

            end_adq = time.time()  # Detiene el temporizador
            tiempo_adquisicion = end_adq - start_adq
            print(n,f"- acquisition time: {tiempo_adquisicion} seconds")

            # Check for keyboard input
            key = cv2.waitKey(1) & 0xFF

            # Display the current state in the window
            cv2.imshow("Frame1", Frame1)
            #cv2.imshow("Frame2", Frame2)

            if key == ord('q'):
                break

Thank you very much

Excellent! Thank you very much jakaskerl

Hi @jakaskerl, thank yo very much!

It works with blocking=False, I no longer see those old frames when I switch cameras.

Although, I was trying to avoid using this because it increases a lot (about double) the delay time between capture and display. At the same time, the frame acquisition rate is quite irregular.

I think this is because in the first case (blocking=True) there are already frames available in the buffer to be taken when requested, and in the second case there may or may not be frames available in the buffer, depending on whether or not they were read in time, right?

Hi @jakaskerl , thanks for your help.

It seems that it is not due to the high resolution, because even if I decrease it to 1080_P, although the effect is more difficult to appreciate, because the program runs faster, when I switch camera I still see some past frames until it jumps to the current ones.

I also tried with the command you mention "DEPTHAI_LEVEL=trace python3 script.py", but it's still the same.

Hello everyone, I am having a problem and I would appreciate if you could help me.

I have two modules IXM378 connected to an OAK-FFC-4P module, I want to see on screen only one of the cameras at a time, therefore, when I press the space bar I make a switch to the opposite camera. The problem is that, when I make a switch it shows me 9 frames of the previous moment in which I was using the current camera, instead of showing me directly the current moment, and that there is a fluid transition between both videos. Here is a graphical example to make it clear:

What I find strange is that the buffer is configured with a queue size of 1 frame, that is, all the time the buffer is being updated with the last frame (current moment), so I don't understand where that additional 9 frames from a past moment come from.

Here is the code I am using:

import cv2	
import depthai as dai
import numpy as np

def getFrame(queue):
    # Get frame from queue
    frame = queue.get()
    # Convert frame to OpenCV format and return
    return frame.getCvFrame()


if __name__ == '__main__':

    fps = 3
    current_state = "Framecam1a"

    # Define a pipeline
    pipeline = dai.Pipeline()
    device = dai.Device()


    # DEFINE SOURCES AND OUTPUTS
    cam1a = pipeline.create(dai.node.ColorCamera)
    cam1b = pipeline.create(dai.node.ColorCamera)
    cam1a.setBoardSocket(dai.CameraBoardSocket.CAM_A)  # 4-lane MIPI  IMX378
    cam1b.setBoardSocket(dai.CameraBoardSocket.CAM_D)  # 4-lane MIPI  IMX378
    cam1a.setResolution(dai.ColorCameraProperties.SensorResolution.THE_12_MP)
    cam1b.setResolution(dai.ColorCameraProperties.SensorResolution.THE_12_MP)
    cam1a.setFps(fps)
    cam1b.setFps(fps)

    # Set output Xlink 
    outcam1a = pipeline.create(dai.node.XLinkOut)
    outcam1b = pipeline.create(dai.node.XLinkOut)
    outcam1a.setStreamName("cam1a")
    outcam1b.setStreamName("cam1b")
      
    # LINKING
    cam1a.isp.link(outcam1a.input)
    cam1b.isp.link(outcam1b.input)

    # Pipeline is defined, now we can connect to the device
    with device:
        device.startPipeline(pipeline)
        # Get output queues.
        cam1a = device.getOutputQueue(name="cam1a", maxSize=1)
        cam1b = device.getOutputQueue(name="cam1b", maxSize=1)
            
        while True:

             # Acquire frames only for the current state
            if current_state == "Framecam1a":
                Frame = getFrame(cam1a)
            else:
                Frame = getFrame(cam1b)

            # Display the current state in the window
            cv2.imshow("Frame", Frame)

            # Check for keyboard input
            key = cv2.waitKey(1) & 0xFF

            if key == ord(' '):
                n=0
                # Change the state
                if current_state == "Framecam1a":
                    current_state = "Framecam1b"
                else:
                    current_state = "Framecam1a"

            if key == ord('q'):
                break

Hello!

I wonder if there is any way to flash the EEPROM manually instead of with calibrate.py

I am using an OAK-FFC 4P with two IMX378 modules, I have run calibrate.py and the EEPROM has been flashed. I have to prepare several of these cameras (OAK-FFC 4P + 2x IMX378), the extrinsic values are the same for all cameras, so I wonder if it is possible to flash manually those others, with the intrinsic values of the first flashed camera, instead of running calibrate.py too many times with each camera.

If possible, what intrinsic values should I get from the first flashed camera and how do I get them? Additionally, how do I flash those values obtained from the first camera to the rest of the cameras?

Thanks!

jakaskerl

Create an RGB cam with high resolution, then output one end directly to xlink node and the other end (preview) to stereo depth node (along with the other RGB camera - also preview output).

When I do this, I get the following error:

.

There is cam_d queue missing... Since you have created the Xlink node, but it's queue can't be read, it fill's up, backfeeding the rest of the pipeline and blocking it.

There were two problems, one, the one you mention, and the other, the enableRectified option. Even adding the cam_d queue or even deleting the whole cam_d output node it still froze. It was solved by disabling this enableRectified option and adding the cam_d queue or deleting the cam_d output node (since I only need the output of one of the two RGB images and the stereo output).