Measurement error XY coordinates

emily

I am performing a simple test measuring a point in space and although the depth indicates it correctly I observe errors in the measurement of X and Y. For example, a point located at X= 0mm, Y=160mm,
Z = 510 mm and gives me a measurement of X= -7, Y = 196, Z = 509.... What could I be configuring wrong?
Thank you in advance for any guidance you can give me 🙂

erik

Hi emily ,
From my understanding, you have an object at 51cm (which is correct), but the X/Y isn't correct? If so, how off is it (what's the error)?
Thanks, Erik

emily

erik Hi Erik, thank you very much for your reply ...
The error at an real measurement of X= 0mm, Y = 160mm I get a camera measurement of X= -7 and Y=197 the error in Y is larger than the ~ 37 mm, the error is about 1.23% but it is not constant it increases as you move the point to be measured away from the center.

erik

Hi emily ,
What about the Z error? As X/Y are calculated from the Z (see here).
Thanks, Erik

K2_

erik

Hi Erik,

I'm hijacking this comment and thread because there's something I don't get. SpatialLocationCalculator seems to imply that the spatialcoordinates.z value is not depth, but the Z coordinate along the optical axis of the camera.

Yet, when doing the spatial coordinates calculation on host, as you linked to, the Z value returned is exactly the depth. Except along the optical axis, where the sphere of constant depth tangents the plane of constant Z, the values should be different:

depth = sqrt ( X² + Y² + Z²) and not depth = Z

Could you clarify?

emily

Hello,

I have tested with another camera of the same model OAK-D, where The actual dimensions of the point are X = 5mm, Y = 130mm, Z= 509mm. and in this case the depth error is 2 mm and the Y error is about 25mm...

What can I do to improve this result?
Any example of the above but in c++ instead of pyton?

erik

Hi emily ,
Are you measuring X/Y from the centroid of the bounding box? Regarding C++ - not really, but chatgpt works does impressive job at translating from one coding language to another🙂
Thanks, Erik

jakaskerl

Hi K2_
Depth is set as the z coordinate, is calculated from the disparity and is constant in a given xy plane. What SpatialLocationCalculator does, is it calculates x and y based on perceived depth and angle from the center.
X being width, Y being height and Z depth.
Then sqrt ( X² + Y² + Z²) would be distance i guess.

Thanks,
Jaka

K2_

Ok, so I was wrong, my instinctive visualization of depth frames was off, thanks for the clarification !

One last question: why in the linked script HFOV is used to calculate Y, instead of VFOV?

(I also tried to replicate the spatial values given by the spatialdetectionnetwork using the linked formulas (using Z and either VFOV or HFOV of RGB camera): it works for X, but but Y is off for now (a multiplicative factor). I don't understand why so far, I'll get back here if I find more information. I have a square 480*480 frame, and keepaspectratio is false)

jakaskerl

Hi K2_
It wouldn't make a difference. Angle calculation uses the following formula:
math.atan(math.tan(HFOV / 2.0) * offset / (frame.shape[1] / 2.0))
This means distance along horizontal axis cancels out, so you are left with atan(offset/depth). Same would happen if we were to use the VFOV and frame.shape[0].

Hope this helps,
Jaka

K2_

Taking 1080p and oak d lite for values:

Tan(54/2)/(1080/2)= 9.43 E-4

Tan(69/2)/(1920/2)=7.16 E-4

I must be missing something, because that doesn't seem to be interchangeable..

jakaskerl

Hi K2_
Values you have sent are for color camera, where 1080p resolution doesn't capture the whole FOV (source) since it's 16/9 while full resolution 4208x3120 is 4/3.
Camera specs for oak d lite:

With above values you should calculate angle to be very similar for both HFOV and VFOV (some discrepancy due to rounded FOV angles).

Hope this helps,
Jaka

K2_

Thanks, and sorry for being a drag! With that I found that the multiple I had off when calculating Y on host is 9/16, so it's at least a number that probably doesn't come from nowhere.

Your answer gets me a new question though: does camRgb.setPreviewKeepAspectRatio(False)capture full ISP FOV to preview, or just the FOV of the cropped 4K from video?

Thanks again !

jakaskerl

Hi K2_
Well, you had me seriously doubting my math before I realized what's going on 😉.
camRgb.setPreviewKeepAspectRatio(False) captures the FOV from video.

Thanks,
Jaka

K2_

Hi, and thanks again for helping. But then isn't this page misleading?
https://docs.luxonis.com/projects/api/en/latest/tutorials/maximize_fov/#maximizing-fov

A challenge occurs when your NN model expects a different aspect ratio (eg. 1:1) compared to the sensors aspect ratio (eg. 4:3), and we want to run NN inference on the full FOV of the sensor. Let’s say we have a MobileNet-SSD that requires 300x300 frames (1:1 aspect ratio) - we have a few options:

Stretch the ISP frame to 1:1 aspect ratio of the NN

The link for "1 Stretch the ISP frame" leads to the section below:

Change aspect ratio

Pros: Preserves full FOV. Cons: Due to stretched frames, NNs accuracy might decrease.

Changing aspect ratio (stretching) can be used Use camRgb.setPreviewKeepAspectRatio(False).

So this reads as setPreviewKeepAspectRatio false is preserving full FOV, by stretching ISP… Especially since on the picture it shows an ISP titled window and a "stretched frame" titled window which seem to have the same FOV, which is not true?

The gen2-full-fov-nn/api/stretching_api.py example scales down ISP (camRgb.setIspScale(1,5) # 4056x3040 -> 812x608) then uses a preview of the same size, which is then stretched to 1:1 through an imagemanip.

What's the shape of the video stream in that case? 12MP 4:3? 812x456 (a cropped 16:9 from 812x608) ? 812x608 4:3?

If preview is derived from video, I guess it's the last one, otherwise you're not full ISP fov. And the framerate, according to the example, has to be limited to 25FPS, because you're running @12MP.

So either you're not getting the FOV advertised (if you're just stretching video which is cropped), or you're not getting the FPS advertised (to take the extreme: 1080p @ 60FPS). But maybe I missed something again?

jakaskerl

Hi K2_
The shape of the video stream is the same as the scaled ISP (meaning 812x608 4:3). If you wish to run at higher FPS you will have to lower camera resolution and in turn, you won't get the full FOV. This seems to be the case currently, yes.

Thanks,
Jaka