Hi, I would like to have a question regarding the precision of obtained depth from **OAK-D Pro W.** From your website https://docs.luxonis.com/software/depthai-components/nodes/stereo_depth/ we can read, that the depth is calculated in 16bit values. That is probably due to the fact, that it is outputed in millimeters, right? Moreover, I have noticed that the depth to objects often jumps, e.g. by 5 cm or more. I also tried to turn on the Subpixeling and it immediately was more precise, like 1 cm. So, my question is, what is the cause? My idea is, that the depth is originaly in 8 bit (0-255) and when converted to 16bit (for mm), it causes that between each step in depth, it jumps by that 5 cm? And as it is described in the following picture, when subpixel is ON, it is originally in 16bit, which means the precision is much higher than in 8 bit? [upl-image-preview url=https://discuss.luxonis.com/assets/files/2024-10-17/1729153950-511676-image.png] To explain my question, my case is that I have to have a camera in 12MP and 800p depth constantly at 30 FPS, which does not allow me to turn on subpixel, because then the camera drops to 10 FPS. However, without subpixel the depth is not so accurate, it jumps by many centimeters. At the same time, I would like to ask whether it is possible to get the original depth directly in 16bit depth without having to turn on the subpixel? Thank you a lot.

Hi @"VojtechVolprecht"#p26573 > @"VojtechVolprecht"#p26573 That is probably due to the fact, that it is outputed in millimeters, right? UINT16 offers higher percision, which UINT8 can't give. If disparity is 10 and depth is 10450mm, this won't be covered by UINT8. > @"VojtechVolprecht"#p26573 Moreover, I have noticed that the depth to objects often jumps, e.g. by 5 cm or more. I also tried to turn on the Subpixeling and it immediately was more precise, like 1 cm. > > So, my question is, what is the cause? There are 95 disparity values by default, so one step could result in a large jump in actual depth - see chart [here](https://docs.google.com/spreadsheets/d/1ymn-0D4HcCbzYP-iPycj_PIdSwmrLenlGryuZDyA4rQ/edit?gid=0#gid=0) When enabling subpixel, this disparity range increases - [docs](https://docs.luxonis.com/hardware/platform/depth/configuring-stereo-depth/#Configuring%20Stereo%20Depth-3.%20Improving%20depth%20accuracy-Stereo%20Subpixel%20mode-Stereo%20subpixel%20effect%20on%20layering) so this effect is minimized. > @"VojtechVolprecht"#p26573 However, without subpixel the depth is not so accurate, it jumps by many centimeters. You can do a disparity shift if your depth is constantly at some range to increase the disparity there. Thansk Jaka

Hi @"jakaskerl"#1112 Thanks a lot. So if we focus on depth in range 1.5m to 3.5m, can we somehow increase the precision here? Or how to calculate the shift?

@"VojtechVolprecht"#p26596 You can increase the precision for values lower than current minZ... [docs](https://docs.luxonis.com/hardware/platform/depth/configuring-stereo-depth/#Configuring%20Stereo%20Depth-4.%20Short%20range%20stereo%20depth-How%20to%20get%20lower%20MinZ-Disparity%20shift). So for 1.5 to 3.5, the precision is still the same and comes from basic trigonometry of perspective. Only way to increase it there is subpixel unfortunately. Thanks, Jaka

Aah. Okay, thanks a lot for the information.

Since the subpixel is a quite huge operation which costs a lot of resources. Can we somehow do it in chosen ROI? I can imagine, that it would lower the demand if we say, that subpixeling iterests us in 50x50 window?

@"VojtechVolprecht"#p26618 No, that is currently not possible. If you have scarce resource, consider using a smaller left and right image (like a 300x300 crop), and turn on the subpixel. Thanks, Jaka

Thanks for the info :-) Hopefully last question :D We use the Pro Wide camera so in order to align the RGB and Depth, we had to use at first the manual undistortion via OpenCV: https://docs.luxonis.com/software/perception/rgb-d. We came across problematic behaviour. There is some discrepancy between measured depth values between this setting and setting when Alpha scaling is not set. If I understand it correctly, we ought to use the Alpha scaling, since we want 100% align with undistorted RGB and its Depth. However, if we just undistort and not scale via Alpha, the measured distances jump around our needed 4-5cm and not 7-10cm like with the set scaling. Is the scaling set by some default value? Below is the code we used, plus without scaling, we just commented out the setAlphaScaling(). ``` alpha = 0 stereo.setAlphaScaling(alpha) rgb_w = camRgb.getResolutionWidth() rgb_h = camRgb.getResolutionHeight() rgbIntrinsics = np.array(calibData.getCameraIntrinsics(rgbCamSocket, rgb_w, rgb_h)) rgb_d = np.array(calibData.getDistortionCoefficients(rgbCamSocket)) rgb_new_cam_matrix, _ = cv2.getOptimalNewCameraMatrix(rgbIntrinsics, rgb_d, (rgb_w, rgb_h), alpha) map_x, map_y = cv2.initUndistortRectifyMap(rgbIntrinsics, rgb_d, None, rgb_new_cam_matrix, (rgb_w, rgb_h), cv2.CV_32FC1) frameRgb = cv2.remap(frameRgb, map_x, map_y, cv2.INTER_LINEAR) ```

@"VojtechVolprecht"#p26675 Alpha for depthai is calculated based on left frame intrinsics so it is not a "default" value. In order to align the images, the alpha must be the same in both cases. Thanks Jaka

Thanks. I understand, but once I set the Alpha scaling to zero, the depth values jumps by average more than 10 cm. If I wont use Alpha scaling, meaning I commented that line out, the depth values are more precise. However, as you also mentioned, without the alpha scaling, the depth is not perfectly aligned to RGB. That is what I worry most about, because we must use undistortion upon our RGB, since we use the Pro Wide camera. We are doing that by your tutorial with OpenCV and its manual undistortion. We set its matrix Alpha the same way we set the Alpha for stereo alphascaling. This is done for the perfect alignment. However, the depth values after that jumps more. We dont know why.

@"VojtechVolprecht"#p26702 That shouldn't be the case since values are not changed, only the image is warped... Can you provide a MRE? Thanks, Jaka

Depth 16bit vs 8bit

VojtechVolprecht

jakaskerl Can you elaborate on what you wish to achieve?

Yes, we are thinking of "subpixeling" in specific ROI on host PC. Because we need to boost the accuracy and the subpixeling look good, but it cost a lot of resources. We still need to proceed in 30 FPS, therefore, we are thinking of subpixeling just important ROI.

jakaskerl

VojtechVolprecht
AFAIK, cost debug output was not added in the API, likely due to the immense message size. I doubt it would be of any help anyway.

Thanks,
Jaka

« Previous Page