Resolutions - ColorCamera + Depth + ISP scaling

VojtechVolprecht

Hi, are there any description of possible input resolutions? I found this table, where are described few of them, but when I look inside the code (ColorCameraProperties.hpp) I find more SensorResolution:

My goal is to have setting where I can align color camera with depth, but I dont know, what are my options. Moreover, I need to know the ISP scaling params for it like it is in the upper table. I want to try e.g. the 1080p option but the only way is to guess the ISP scale (e.g. 1,1) in order to align with depth.

Thx for info.

jakaskerl

Hi VojtechVolprecht
Not all resolutions are supported by all sensors. Colorcamera can usually only go down to 1080p for IMX378 but can go lower for OV9782.

For ISP scale, you can use this script:


import math
lst = []
for n in range(1, 16+1):
    for d in range (n, 63+1):
        # Chroma needs 2x extra downscaling
        if d < 32 or n % 2 == 0:
            # Only if irreducible
            if math.gcd(n, d) == 1:
                item = (n/d, (n, d))
                lst.append(item)
lst.sort(reverse=True)
for e in lst: print(f'{e[0]}, {e[1][0]}, {e[1][1]}')

first calculate the ratio, then find the closest scale.

Thanks,
Jaka

VojtechVolprecht

Okay, thx.
I am wondering now. I have the OAK pro wide with imx378. The original resolution is then 12MP. So what happens, when I set resolution for ColorCamera node to 1080p? Does it mean, that the 12MP is cropped to 1080p? And then second question, what happens if I further use ISP scaling on such 1080p?

Moreover, is there some description of possible FPS configuration? I found really confusing behaviour, that when 30 FPS is set, everything runs in 30 FPS, but when I set 60 FPS, it drops to 12 FPS.

Thanks a lot.

jakaskerl

VojtechVolprecht
When you set the resolution for the ColorCamera node to 1080p on your OAK Pro Wide with the IMX378 sensor, the ISP (Image Signal Processor) first crops the full 3:4 size to 16:9 (4K - and you lose some FOV), then scales down the image to 1080p. This is not a crop, but a downscale, meaning the entire field of view is preserved, but the image is resized to a lower resolution. docs

Moreover, is there some description of possible FPS configuration? I found really confusing behaviour, that when 30 FPS is set, everything runs in 30 FPS, but when I set 60 FPS, it drops to 12 FPS.

Could be related to this

Thanks,
Jaka

VojtechVolprecht

Thx for the docs.

May I further ask you, since I use the Pro Wide, we have to undistort the image. I found out, that in 2.28.0 depthai-core library, there is some undistortion for ToF sensors (not for stereo). I also found, that with using Camera node instead of ColorCamera, there should be undistortion by default. However, with Camera node, there is also different resolution setting than for ColorCamera. Instead of ISP scaling, there is setting for width & height. I tried to set the resolution similar to ISP scaling, but it only outputs errors. I tried the common 16:9 resolutions like 1080p and that works. However, I need also 4:3 resolutions and I did not find any possbile setting, that would not output error. Unfortunately, I did not find any documentation about such possible configurations.

So my question is, are there any docs about such resolutions for Camera node? Are there any 4:3 resolutions? Is the Camera node fully configurable like ColorCamera?

Thx for the help.

jakaskerl

VojtechVolprecht
cam.setSize() to a 4:3 width and height. Then the Camera node should automatically set appropriate resolution and scaling to minimize crop.
It still has the same outputs as the ColorCamera node.

Thanks,
Jaka

VojtechVolprecht

That is pretty interesting, because when I set Camera node to 1248x936 which is exactly after the ISP scaling I had before for ColorCamera, now the Camera node outputs error during aligning with depth with info, that the RGB camera width is 1352, which is not what I set.

I think, that the Camera node automatically set some other appropriate resolution, but such that it is not viable with aligning to depth.

Do you have any suggestions?

jakaskerl

VojtechVolprecht
That is because IMX378 is capable of having RAW sensor resolution as 1352x1014 and since this is the closest to your desired res, this one is selected.

You need to set setVideoSize() in order to have video output at the desired W/H. ISP output is bound to sensor resolution + ISP SCALE.

Thanks,
Jaka