gen2-triangulation inaccurate depth

Arthur

When I run gen2-triangulation on my Oak-D pro, I get very inaccurate coordinates. I positioned the device at eye level and positioned my face 1m away from it using a measuring stick, and the text in the visualization window says my face (from looking at the code it's actually the first detected landmark) is only 80cm away. When I run gen2-calc-spatials-on-host the results are accurate within +-1cm at the same distance.

I am using Python3.8. Here are the packages that are installed in my virtual environment as a result of running pip install -r requirements.txt:

Package                   Version
------------------------- ---------
addict                    2.4.0
asttokens                 2.4.1
attrs                     23.2.0
backcall                  0.2.0
blinker                   1.7.0
blobconverter             1.2.8
boto3                     1.17.39
botocore                  1.20.112
certifi                   2024.2.2
charset-normalizer        3.3.2
click                     8.1.7
comm                      0.2.1
ConfigArgParse            1.7
contourpy                 1.1.1
cycler                    0.12.1
dash                      2.15.0
dash-core-components      2.0.0
dash-html-components      2.0.0
dash-table                5.0.0
decorator                 5.1.1
depthai                   2.17.4.0
executing                 2.0.1
fastjsonschema            2.19.1
Flask                     3.0.2
fonttools                 4.48.1
idna                      3.6
importlib-metadata        7.0.1
importlib-resources       6.1.1
ipython                   8.12.3
ipywidgets                8.1.2
itsdangerous              2.1.2
jedi                      0.19.1
Jinja2                    3.1.3
jmespath                  0.10.0
joblib                    1.3.2
jsonschema                4.21.1
jsonschema-specifications 2023.12.1
jupyter_core              5.7.1
jupyterlab_widgets        3.0.10
kiwisolver                1.4.5
MarkupSafe                2.1.5
matplotlib                3.7.4
matplotlib-inline         0.1.6
nbformat                  5.9.2
nest-asyncio              1.6.0
numpy                     1.24.4
open3d                    0.18.0
opencv-python             4.5.1.48
packaging                 23.2
pandas                    2.0.3
parso                     0.8.3
pexpect                   4.9.0
pickleshare               0.7.5
pillow                    10.2.0
pip                       24.0
pkgutil_resolve_name      1.3.10
platformdirs              4.2.0
plotly                    5.18.0
prompt-toolkit            3.0.43
ptyprocess                0.7.0
pure-eval                 0.2.2
Pygments                  2.17.2
pyparsing                 3.1.1
pyquaternion              0.9.9
python-dateutil           2.8.2
pytz                      2024.1
PyYAML                    6.0.1
referencing               0.33.0
requests                  2.31.0
retrying                  1.3.4
rpds-py                   0.17.1
s3transfer                0.3.7
scikit-learn              1.3.2
scipy                     1.10.1
setuptools                56.0.0
six                       1.16.0
stack-data                0.6.3
tenacity                  8.2.3
threadpoolctl             3.2.0
tqdm                      4.66.2
traitlets                 5.14.1
typing_extensions         4.9.0
tzdata                    2024.1
urllib3                   1.26.18
wcwidth                   0.2.13
Werkzeug                  3.0.1
widgetsnbextension        4.0.10
zipp                      3.17.0

jakaskerl

Hi @Arthur
Check the line luxonis/depthai-experimentsblob/master/gen2-triangulation/main.py#L263. Make sure the scaling factors are the same for your case. It might be that due to different W/H or aspect ration, the disparity is calculated incorrectly.

Thanks,
Jaka

Arthur

Thanks for your help jakaskerl I have verified that that the resize_factor variable is 2.4 (720 / 300) as it should be according to the comments. I'm not sure why the resolution of the images should vary though. I have not modified the code in any way.

jakaskerl

Hi @Arthur
Could happen that due to unsupported resolution, a different default resolution of the camera is set. Likely not the case since you are on the D-Pro. I have tested the script without installing the requirements and it worked relatively well (relatively because the depth is not as accurate of course).

Could you add some images of the error perhaps? Maybe a piece of paper with a face and ground truth distance?

Thanks,
Jaka

Arthur

jakaskerl Here is the setup with the camera 1m away from the laptop screen displaying an image and a measurement of 79cm in the output window.

Arthur

jakaskerl What kind of accuracy did you get? 20% error is not normal right?

I also noticed in the code that the images are not rectified before being fed into the neural net.

jakaskerl

Hi @Arthur
Right, are you using a WIDE variant of the D-Pro? I got about 10% error I think, but not sure how accurate the distance I measure was. 20% is a bit too much anyway.

Thanks,
Jaka

Arthur

jakaskerl No it's a regular Oak-D pro with fixed focus. How come the error is so much higher than with regular depth?

jakaskerl

Hi @Arthur
I'd say it's likely due to no rectification being done. Have you tried feeding it rectified images?

Thanks,
Jaka