I am a bit confused on the calculation of the WRT disparity shift. Please help! * Camera OAK-D Pro W PoE OV9782 * The min/max depth distance is \~300/700 (mm) * With 400P mono camera resolution * Extended disparity: true What value of the disparity shift should be?

Thanks @"erik"#p11832 ! I tried with setting setDisparityShift() 90 resp. 95, mono 800P and below settings [upl-image-preview url=https://discuss.luxonis.com/assets/files/2023-09-11/1694416819-580528-image.png] It still spreads a lot from the targets (yellow marked, distance: about 360 mm - 610 mm) How can we improve that? [URL=https://i.imgur.com/GH4VtnB.png][IMG]https://i.imgur.com/GH4VtnB.png[/IMG][/URL]

Hi @"[unknown]"#p11883 , This doesn't seem like a problem with disparity shift, perhaps mapping detection to depth map (to get Z coordinate of the object) is the problem here. Could you double-check that first?

[upl-image-preview url=https://discuss.luxonis.com/assets/files/2023-09-12/1694542447-789315-image.png] [upl-image-preview url=https://discuss.luxonis.com/assets/files/2023-09-12/1694542508-267704-image.png]

I notice one thing, my camera doesn't have the same y-pixel coordinate between left and right mono camera image!?! (The square is 50 mm x 50 mm) I run the [stereo neural inference demo](https://github.com/luxonis/depthai-experiments/tree/master/gen2-triangulation#triangulation---stereo-neural-inference-demo) as it is. [upl-image-preview url=https://discuss.luxonis.com/assets/files/2023-09-12/1694560633-402852-image.png] Not as seen here [upl-image-preview url=https://user-images.githubusercontent.com/59799831/132098832-70a2d0b9-1a30-4994-8dad-dc880a803fb3.gif]

WRT disparity shift

kle

I am a bit confused on the calculation of the WRT disparity shift. Please help!

Camera OAK-D Pro W PoE OV9782
The min/max depth distance is ~300/700 (mm)
With 400P mono camera resolution
Extended disparity: true

What value of the disparity shift should be?

erik

Hi kle ,
You should check full-pixel stereo calc:

So if you are using 800P OV9782 (which has focal distance about 880 pixels), you can set disparity shift to 90, so MaxZ would be about 73.5 cm, and MinZ would be 90 (shift) + 95 (pixel search) => 185 pixels, so about 38cm.

If you are using 400P, then you can halve the focal distance, and check again what's the max value you can select so distance will still be above your MaxZ (70cm).

kle

Thanks erik !

I tried with setting setDisparityShift() 90 resp. 95, mono 800P and below settings

It still spreads a lot from the targets (yellow marked, distance: about 360 mm - 610 mm)
How can we improve that?

kle

Here are 2 depth images with bounding box mapping.
The second one shows all object boxes are not accurate, which probably results the incorrect spatial coordinate values.
(note: Which it "fails", it always fails for all boxes)

Why and how to correct/prevent that?

erik

Hi kle ,
Could you also overlay color frame on top of it? As it's not entirely clear just from the depth map where the target objects actually are.

kle

I notice one thing, my camera doesn't have the same y-pixel coordinate between left and right mono camera image!?!
(The square is 50 mm x 50 mm)
I run the stereo neural inference demo as it is.

Not as seen here

kle

Below is a screenshot when below data for the same object and same detection doesn't matched!

detection.boundingBoxMapping.roi[topLeft, bottomRight] (blue box)
detection.[xmin, ymin, xmax, ymax] (green box)

The outcome is below
"D" : Blue box
"C": Green box
"==>": Diff
=== 1 === === 2 === D {922, 468, 944, 508} C {923, 563, 945, 638} ==> {-1, -95, -1, -130} D {574, 227, 585, 274} C {574, 111, 585, 200} ==> {0, 116, 0, 74} D {259, 460, 284, 502} C {259, 549, 284, 627} ==> {0, -89, 0, -125} D {323, 267, 337, 303} C {324, 187, 338, 253} ==> {-1, 80, -1, 50} D {914, 268, 927, 307} C {915, 189, 927, 261} ==> {-1, 79, 0, 46} === 3 === D {922, 468, 944, 509} C {922, 469, 944, 510} ==> {0, -1, 0, -1} D {259, 460, 284, 502} C {260, 461, 285, 503} ==> {-1, -1, -1, -1} D {574, 227, 585, 273} C {575, 228, 586, 274} ==> {-1, -1, -1, -1} D {324, 267, 338, 302} C {324, 268, 338, 303} ==> {0, -1, 0, -1} D {914, 269, 927, 309} C {914, 269, 927, 309} ==> {0, 0, 0, 0} === 4 === D {922, 468, 944, 509} C {922, 564, 944, 640} ==> {0, -96, 0, -131} D {574, 226, 585, 274} C {575, 109, 585, 199} ==> {-1, 117, 0, 75} D {260, 461, 285, 502} C {260, 550, 285, 626} ==> {0, -89, 0, -124} D {323, 267, 337, 304} C {323, 187, 337, 256} ==> {0, 80, 0, 48} D {914, 268, 927, 309} C {915, 188, 928, 264} ==> {-1, 80, -1, 45} === 5 === D {922, 468, 944, 508} C {923, 469, 945, 508} ==> {-1, -1, -1, 0} D {574, 226, 585, 273} C {575, 227, 585, 274} ==> {-1, -1, 0, -1} D {323, 267, 337, 305} C {324, 267, 338, 305} ==> {-1, 0, -1, 0} D {259, 461, 285, 502} C {260, 461, 286, 503} ==> {-1, 0, -1, -1} D {914, 268, 927, 308} C {915, 268, 928, 309} ==> {-1, 0, -1, -1} === 6 === D {922, 468, 944, 509} C {922, 469, 944, 510} ==> {0, -1, 0, -1} D {574, 227, 585, 273} C {574, 227, 585, 274} ==> {0, 0, 0, -1} D {259, 460, 284, 502} C {259, 461, 285, 503} ==> {0, -1, -1, -1} D {324, 267, 337, 303} C {324, 267, 338, 303} ==> {0, 0, -1, 0} D {914, 269, 928, 309} C {914, 269, 928, 309} ==> {0, 0, 0, 0} === 7 === D {574, 226, 585, 274} C {574, 111, 585, 200} ==> {0, 115, 0, 74} D {922, 468, 944, 508} C {922, 563, 944, 638} ==> {0, -95, 0, -130} D {258, 460, 284, 502} C {259, 549, 284, 628} ==> {-1, -89, 0, -126} D {324, 267, 338, 303} C {324, 187, 338, 255} ==> {0, 80, 0, 48} D {914, 268, 927, 309} C {914, 189, 928, 265} ==> {0, 79, -1, 44} === 8 === D {922, 468, 944, 509} C {923, 469, 945, 509} ==> {-1, -1, -1, 0} D {259, 460, 283, 502} C {260, 461, 284, 502} ==> {-1, -1, -1, 0} D {574, 226, 585, 274} C {574, 227, 585, 274} ==> {0, -1, 0, 0} D {324, 268, 338, 304} C {324, 268, 338, 305} ==> {0, 0, 0, -1} D {914, 268, 927, 309} C {914, 268, 928, 309} ==> {0, 0, -1, 0} === 9 === D {922, 468, 944, 508} C {923, 564, 945, 638} ==> {-1, -96, -1, -130} D {574, 226, 584, 273} C {574, 110, 585, 198} ==> {0, 116, -1, 75} D {259, 460, 285, 502} C {259, 549, 285, 628} ==> {0, -89, 0, -126} D {324, 267, 337, 302} C {324, 187, 337, 252} ==> {0, 80, 0, 50} D {914, 269, 927, 309} C {915, 191, 928, 265} ==> {-1, 78, -1, 44} === 10 === D {922, 468, 944, 509} C {922, 469, 944, 510} ==> {0, -1, 0, -1} D {259, 460, 284, 502} C {259, 461, 285, 503} ==> {0, -1, -1, -1} D {574, 226, 585, 273} C {574, 227, 585, 274} ==> {0, -1, 0, -1} D {324, 268, 337, 303} C {324, 268, 337, 303} ==> {0, 0, 0, 0} D {914, 269, 927, 309} C {914, 269, 927, 309} ==> {0, 0, 0, 0} === 11 === D {922, 468, 944, 507} C {922, 564, 944, 637} ==> {0, -96, 0, -130} D {259, 460, 284, 502} C {260, 549, 284, 626} ==> {-1, -89, 0, -124} D {574, 226, 585, 273} C {574, 109, 585, 199} ==> {0, 117, 0, 74} D {324, 267, 337, 301} C {325, 186, 338, 250} ==> {-1, 81, -1, 51} D {914, 268, 927, 309} C {914, 189, 928, 266} ==> {0, 79, -1, 43} === 12 === D {922, 468, 944, 509} C {922, 563, 944, 640} ==> {0, -95, 0, -131} D {574, 226, 585, 274} C {574, 110, 585, 199} ==> {0, 116, 0, 75} D {324, 267, 337, 301} C {324, 187, 337, 250} ==> {0, 80, 0, 51} D {259, 460, 283, 501} C {259, 549, 283, 626} ==> {0, -89, 0, -125} D {914, 268, 927, 309} C {914, 188, 927, 265} ==> {0, 80, 0, 44} === 13 === D {574, 226, 585, 273} C {575, 226, 585, 274} ==> {-1, 0, 0, -1} D {922, 468, 944, 509} C {922, 469, 944, 510} ==> {0, -1, 0, -1} D {259, 461, 284, 502} C {260, 461, 285, 502} ==> {-1, 0, -1, 0} D {323, 267, 337, 304} C {323, 268, 337, 305} ==> {0, -1, 0, -1} D {914, 268, 927, 308} C {915, 269, 928, 309} ==> {-1, -1, -1, -1} === 14 === D {922, 468, 944, 510} C {922, 564, 944, 641} ==> {0, -96, 0, -131} D {574, 226, 585, 274} C {575, 109, 586, 199} ==> {-1, 117, -1, 75} D {323, 267, 337, 303} C {323, 187, 337, 254} ==> {0, 80, 0, 49} D {259, 460, 285, 503} C {259, 549, 285, 629} ==> {0, -89, 0, -126} D {914, 269, 927, 309} C {915, 190, 928, 265} ==> {-1, 79, -1, 44} === 15 === D {922, 469, 944, 509} C {922, 564, 944, 639} ==> {0, -95, 0, -130} D {574, 227, 585, 274} C {574, 112, 585, 199} ==> {0, 115, 0, 75} D {259, 460, 284, 502} C {259, 549, 284, 628} ==> {0, -89, 0, -126} D {324, 267, 337, 303} C {325, 187, 338, 253} ==> {-1, 80, -1, 50} D {914, 269, 927, 309} C {914, 189, 928, 265} ==> {0, 80, -1, 44} === 16 === D {922, 468, 944, 510} C {922, 564, 944, 642} ==> {0, -96, 0, -132} D {574, 227, 585, 274} C {574, 112, 585, 199} ==> {0, 115, 0, 75} D {259, 460, 282, 502} C {259, 549, 283, 627} ==> {0, -89, -1, -125} D {323, 267, 337, 303} C {323, 187, 338, 254} ==> {0, 80, -1, 49} D {914, 268, 928, 309} C {914, 189, 928, 265} ==> {0, 79, 0, 44} === 17 === D {922, 468, 944, 509} C {923, 469, 945, 510} ==> {-1, -1, -1, -1} D {574, 227, 585, 273} C {574, 228, 585, 274} ==> {0, -1, 0, -1} D {259, 460, 282, 502} C {260, 461, 283, 503} ==> {-1, -1, -1, -1} D {324, 267, 337, 303} C {325, 268, 338, 304} ==> {-1, -1, -1, -1} D {914, 268, 928, 309} C {914, 268, 928, 309} ==> {0, 0, 0, 0} === 18 === D {922, 468, 944, 509} C {923, 563, 945, 640} ==> {-1, -95, -1, -131} D {574, 227, 585, 274} C {574, 111, 585, 199} ==> {0, 116, 0, 75} D {258, 460, 284, 502} C {259, 548, 285, 627} ==> {-1, -88, -1, -125} D {324, 267, 337, 302} C {325, 187, 338, 252} ==> {-1, 80, -1, 50} D {914, 268, 927, 309} C {914, 188, 927, 265} ==> {0, 80, 0, 44} === 19 === D {922, 468, 944, 509} C {922, 469, 944, 510} ==> {0, -1, 0, -1} D {574, 226, 585, 273} C {575, 226, 586, 273} ==> {-1, 0, -1, 0} D {259, 461, 284, 502} C {259, 461, 284, 502} ==> {0, 0, 0, 0} D {323, 267, 337, 303} C {323, 268, 337, 303} ==> {0, -1, 0, 0} D {914, 269, 927, 309} C {914, 269, 927, 309} ==> {0, 0, 0, 0} === 20 === D {922, 468, 944, 508} C {922, 469, 944, 509} ==> {0, -1, 0, -1} D {574, 226, 584, 274} C {574, 226, 585, 274} ==> {0, 0, -1, 0} D {259, 460, 284, 502} C {259, 461, 284, 503} ==> {0, -1, 0, -1} D {324, 267, 338, 304} C {324, 268, 338, 304} ==> {0, -1, 0, 0} D {914, 269, 928, 309} C {914, 270, 928, 310} ==> {0, -1, 0, -1} === 21 === D {574, 227, 585, 274} C {574, 111, 585, 199} ==> {0, 116, 0, 75} D {922, 468, 944, 509} C {923, 564, 945, 640} ==> {-1, -96, -1, -131} D {259, 461, 284, 502} C {259, 550, 284, 627} ==> {0, -89, 0, -125} D {324, 267, 337, 302} C {324, 186, 337, 252} ==> {0, 81, 0, 50} D {914, 268, 927, 308} C {915, 189, 928, 264} ==> {-1, 79, -1, 44} === 22 === D {922, 468, 944, 508} C {922, 469, 944, 509} ==> {0, -1, 0, -1} D {574, 227, 585, 273} C {575, 227, 585, 274} ==> {-1, 0, 0, -1} D {323, 267, 337, 303} C {324, 268, 338, 304} ==> {-1, -1, -1, -1} D {259, 460, 284, 502} C {259, 461, 285, 503} ==> {0, -1, -1, -1} D {914, 268, 927, 308} C {915, 268, 928, 309} ==> {-1, 0, -1, -1} === 23 === D {574, 226, 585, 274} C {575, 110, 585, 199} ==> {-1, 116, 0, 75} D {922, 468, 944, 510} C {922, 564, 944, 643} ==> {0, -96, 0, -133} D {259, 460, 284, 502} C {260, 549, 285, 628} ==> {-1, -89, -1, -126} D {324, 267, 337, 303} C {324, 187, 337, 253} ==> {0, 80, 0, 50} D {914, 268, 927, 309} C {915, 189, 928, 265} ==> {-1, 79, -1, 44} === 24 === D {922, 468, 944, 509} C {922, 469, 944, 510} ==> {0, -1, 0, -1} D {574, 225, 584, 274} C {575, 225, 585, 274} ==> {-1, 0, -1, 0} D {259, 461, 284, 502} C {260, 461, 285, 502} ==> {-1, 0, -1, 0} D {323, 267, 337, 304} C {323, 267, 337, 304} ==> {0, 0, 0, 0} D {914, 268, 927, 309} C {915, 269, 928, 310} ==> {-1, -1, -1, -1} === 25 === D {922, 468, 944, 508} C {923, 564, 945, 639} ==> {-1, -96, -1, -131} D {259, 460, 284, 502} C {259, 549, 284, 628} ==> {0, -89, 0, -126} D {574, 227, 585, 274} C {574, 112, 585, 199} ==> {0, 115, 0, 75} D {324, 268, 337, 303} C {324, 188, 338, 254} ==> {0, 80, -1, 49} D {914, 269, 927, 309} C {915, 190, 928, 265} ==> {-1, 79, -1, 44} === 26 === D {922, 468, 944, 509} C {922, 564, 944, 640} ==> {0, -96, 0, -131} D {574, 225, 584, 274} C {575, 108, 585, 200} ==> {-1, 117, -1, 74} D {322, 267, 337, 303} C {323, 186, 337, 254} ==> {-1, 81, 0, 49} D {258, 461, 284, 502} C {259, 550, 285, 627} ==> {-1, -89, -1, -125} D {914, 268, 927, 309} C {915, 188, 928, 265} ==> {-1, 80, -1, 44} === 27 === D {922, 468, 944, 508} C {923, 469, 945, 509} ==> {-1, -1, -1, -1} D {574, 227, 585, 274} C {574, 227, 585, 274} ==> {0, 0, 0, 0} D {259, 460, 284, 502} C {259, 461, 284, 503} ==> {0, -1, 0, -1} D {324, 267, 337, 302} C {324, 268, 337, 303} ==> {0, -1, 0, -1} D {914, 269, 927, 308} C {915, 269, 928, 309} ==> {-1, 0, -1, -1} === 28 === D {922, 469, 944, 509} C {922, 564, 944, 640} ==> {0, -95, 0, -131} D {259, 461, 284, 502} C {260, 550, 285, 628} ==> {-1, -89, -1, -126} D {574, 226, 585, 274} C {575, 110, 586, 199} ==> {-1, 116, -1, 75} D {324, 268, 337, 303} C {324, 188, 337, 254} ==> {0, 80, 0, 49} D {914, 269, 928, 309} C {914, 190, 928, 266} ==> {0, 79, 0, 43} === 29 === D {922, 468, 944, 509} C {923, 469, 945, 510} ==> {-1, -1, -1, -1} D {574, 227, 585, 274} C {575, 227, 585, 274} ==> {-1, 0, 0, 0} D {258, 461, 284, 502} C {258, 461, 284, 503} ==> {0, 0, 0, -1} D {323, 267, 337, 302} C {324, 268, 338, 302} ==> {-1, -1, -1, 0} D {914, 268, 928, 309} C {915, 268, 928, 309} ==> {-1, 0, 0, 0} === 30 === D {922, 469, 944, 509} C {922, 469, 944, 509} ==> {0, 0, 0, 0} D {574, 226, 585, 274} C {575, 226, 586, 274} ==> {-1, 0, -1, 0} D {323, 267, 337, 302} C {324, 268, 338, 303} ==> {-1, -1, -1, -1} D {259, 461, 284, 502} C {259, 461, 284, 502} ==> {0, 0, 0, 0} D {915, 268, 927, 309} C {915, 269, 928, 310} ==> {0, -1, -1, -1} === 31 === D {574, 227, 585, 273} C {574, 227, 585, 274} ==> {0, 0, 0, -1} D {922, 468, 944, 509} C {923, 469, 945, 510} ==> {-1, -1, -1, -1} D {324, 268, 337, 301} C {324, 268, 337, 302} ==> {0, 0, 0, -1} D {259, 460, 285, 502} C {259, 461, 285, 503} ==> {0, -1, 0, -1} D {914, 269, 927, 309} C {915, 269, 928, 309} ==> {-1, 0, -1, 0} === 32 === D {574, 227, 584, 273} C {574, 228, 585, 274} ==> {0, -1, -1, -1} D {922, 469, 944, 510} C {922, 469, 944, 510} ==> {0, 0, 0, 0} D {257, 460, 284, 502} C {258, 461, 285, 503} ==> {-1, -1, -1, -1} D {324, 267, 337, 303} C {325, 268, 338, 304} ==> {-1, -1, -1, -1} D {914, 269, 927, 308} C {915, 269, 928, 309} ==> {-1, 0, -1, -1} === 33 === D {574, 226, 585, 274} C {574, 227, 585, 274} ==> {0, -1, 0, 0} D {922, 468, 944, 509} C {922, 469, 944, 509} ==> {0, -1, 0, 0} D {258, 460, 284, 502} C {259, 461, 285, 503} ==> {-1, -1, -1, -1} D {323, 267, 337, 304} C {323, 268, 337, 304} ==> {0, -1, 0, 0} D {915, 269, 927, 309} C {915, 269, 928, 309} ==> {0, 0, -1, 0} === 34 === D {922, 468, 944, 510} C {923, 469, 945, 510} ==> {-1, -1, -1, 0} D {574, 225, 585, 273} C {574, 226, 585, 274} ==> {0, -1, 0, -1} D {259, 461, 284, 502} C {260, 461, 285, 502} ==> {-1, 0, -1, 0} D {323, 267, 337, 304} C {323, 268, 337, 305} ==> {0, -1, 0, -1} D {915, 268, 928, 309} C {915, 269, 928, 310} ==> {0, -1, 0, -1} === 35 === D {922, 468, 944, 510} C {923, 469, 945, 510} ==> {-1, -1, -1, 0} D {574, 226, 585, 273} C {575, 226, 586, 274} ==> {-1, 0, -1, -1} D {323, 267, 337, 304} C {324, 267, 338, 304} ==> {-1, 0, -1, 0} D {258, 461, 284, 504} C {259, 462, 285, 505} ==> {-1, -1, -1, -1} D {915, 269, 927, 306} C {915, 269, 928, 306} ==> {0, 0, -1, 0} === 36 === D {922, 468, 944, 509} C {923, 469, 945, 510} ==> {-1, -1, -1, -1} D {574, 227, 585, 274} C {574, 228, 585, 274} ==> {0, -1, 0, 0} D {258, 460, 283, 502} C {259, 461, 283, 503} ==> {-1, -1, 0, -1} D {323, 267, 337, 302} C {324, 268, 337, 302} ==> {-1, -1, 0, 0} D {914, 269, 927, 310} C {915, 269, 928, 310} ==> {-1, 0, -1, 0} === 37 === D {922, 468, 944, 509} C {923, 469, 945, 510} ==> {-1, -1, -1, -1} D {574, 226, 585, 274} C {574, 226, 585, 274} ==> {0, 0, 0, 0} D {259, 461, 284, 502} C {259, 462, 284, 503} ==> {0, -1, 0, -1} D {914, 268, 927, 308} C {915, 269, 928, 309} ==> {-1, -1, -1, -1} D {323, 267, 337, 304} C {323, 268, 337, 304} ==> {0, -1, 0, 0} === 38 === D {922, 469, 944, 509} C {923, 565, 945, 641} ==> {-1, -96, -1, -132} D {574, 227, 585, 274} C {575, 111, 586, 199} ==> {-1, 116, -1, 75} D {258, 460, 284, 502} C {258, 549, 284, 628} ==> {0, -89, 0, -126} D {324, 268, 337, 301} C {324, 188, 337, 251} ==> {0, 80, 0, 50} D {914, 269, 927, 309} C {915, 190, 928, 265} ==> {-1, 79, -1, 44} === 39 === D {922, 469, 944, 510} C {923, 469, 945, 511} ==> {-1, 0, -1, -1} D {574, 227, 585, 274} C {574, 227, 585, 274} ==> {0, 0, 0, 0} D {258, 460, 284, 502} C {258, 461, 284, 503} ==> {0, -1, 0, -1} D {324, 268, 337, 304} C {324, 268, 337, 304} ==> {0, 0, 0, 0} D {915, 269, 927, 308} C {915, 270, 928, 309} ==> {0, -1, -1, -1} === 40 === D {922, 468, 944, 509} C {922, 469, 944, 510} ==> {0, -1, 0, -1} D {574, 227, 585, 274} C {574, 227, 585, 274} ==> {0, 0, 0, 0} D {258, 460, 283, 502} C {259, 461, 284, 503} ==> {-1, -1, -1, -1} D {324, 267, 337, 301} C {324, 268, 337, 302} ==> {0, -1, 0, -1} D {914, 269, 927, 309} C {915, 270, 928, 310} ==> {-1, -1, -1, -1} === 41 === D {574, 227, 585, 274} C {575, 227, 586, 274} ==> {-1, 0, -1, 0} D {922, 468, 944, 510} C {923, 469, 945, 511} ==> {-1, -1, -1, -1} D {259, 461, 284, 502} C {259, 461, 284, 502} ==> {0, 0, 0, 0} D {914, 269, 927, 309} C {915, 269, 928, 309} ==> {-1, 0, -1, 0} D {322, 267, 337, 303} C {323, 268, 338, 303} ==> {-1, -1, -1, 0} === 42 === D {923, 468, 944, 508} C {923, 564, 944, 638} ==> {0, -96, 0, -130} D {574, 227, 585, 274} C {575, 112, 586, 199} ==> {-1, 115, -1, 75} D {259, 460, 283, 502} C {259, 549, 283, 626} ==> {0, -89, 0, -124} D {323, 268, 337, 303} C {324, 188, 338, 253} ==> {-1, 80, -1, 50} D {914, 268, 927, 309} C {915, 188, 928, 265} ==> {-1, 80, -1, 44} === 43 === D {922, 468, 944, 510} C {923, 469, 945, 510} ==> {-1, -1, -1, 0} D {574, 226, 585, 274} C {574, 226, 585, 274} ==> {0, 0, 0, 0} D {258, 461, 284, 502} C {259, 462, 285, 503} ==> {-1, -1, -1, -1} D {323, 267, 337, 304} C {324, 268, 338, 305} ==> {-1, -1, -1, -1} D {915, 269, 928, 308} C {915, 269, 928, 308} ==> {0, 0, 0, 0} === 44 === D {922, 468, 944, 509} C {923, 469, 945, 510} ==> {-1, -1, -1, -1} D {574, 227, 585, 273} C {574, 227, 585, 274} ==> {0, 0, 0, -1} D {323, 267, 337, 302} C {324, 268, 338, 303} ==> {-1, -1, -1, -1} D {259, 461, 282, 501} C {259, 461, 283, 501} ==> {0, 0, -1, 0} D {914, 268, 927, 309} C {915, 269, 928, 310} ==> {-1, -1, -1, -1} === 45 === D {922, 469, 944, 509} C {923, 469, 945, 509} ==> {-1, 0, -1, 0} D {574, 226, 585, 274} C {574, 227, 585, 274} ==> {0, -1, 0, 0} D {322, 268, 337, 306} C {323, 268, 337, 306} ==> {-1, 0, 0, 0} D {259, 460, 283, 501} C {259, 461, 283, 502} ==> {0, -1, 0, -1} D {915, 268, 927, 309} C {915, 269, 928, 309} ==> {0, -1, -1, 0}

The code is
`
int yDelta = (depthFrameColor.rows - nnFrame.rows) / 2;

            for(const auto& detection : detections) {
                auto roiData = detection.boundingBoxMapping;
                auto roi = roiData.roi;
                roi = roi.denormalize(depthFrameColor.cols, depthFrameColor.rows);
                auto topLeft = roi.topLeft();
                auto bottomRight = roi.bottomRight();
                int xmin = (int)topLeft.x;
                int ymin = (int)topLeft.y;
                int xmax = (int)bottomRight.x;
                int ymax = (int)bottomRight.y;
                cv::rectangle(depthFrameColor, cv::Rect(cv::Point(xmin, ymin), cv::Point(xmax, ymax)), constant::COLOR_BLUE, 2);

                int xmin2 = (int)(detection.xmin * nnFrame.cols);
                int ymin2 = (int)(detection.ymin * nnFrame.rows + yDelta);
                int xmax2 = (int)(detection.xmax * nnFrame.cols);
                int ymax2 = (int)(detection.ymax * nnFrame.rows + yDelta);
                cv::rectangle(depthFrameColor, cv::Rect(cv::Point(xmin2, ymin2), cv::Point(xmax2, ymax2)), constant::COLOR_GREEN, 1);


                std::cout << " D {"
                    << xmin2 << ", "
                    << ymin2 << ", "
                    << xmax2 << ", "
                    << ymax2
                    << "}  C {"
                    << xmin << ", "
                    << ymin << ", "
                    << xmax << ", "
                    << ymax
                    << "} ==> {"
                    << xmin2 - xmin << ", "
                    << ymin2 - ymin << ", "
                    << xmax2 - xmax << ", "
                    << ymax2 - ymax
                    << "}" << std::endl
                ;
            }

erik

Hi kle ,
That can happen, and that's what rectification process fixes - it rotates both images so stereo matching can work. Regarding the 2 images - on the below one, where are mappings of the detections?

kle

Hi erik ,
-> Regarding the 2 images - on the below one, where are mappings of the detections
Do you mean the blue box and green box of each detected object on the image?
Then the green box (with size 1) is on top of the blue one (with size 2).

erik

Hi kle ,
I see, that seems the root cause of the issue then - as you have used scale factor of 1, node will take into account background depth (see purple arrows) for spatial coordiante estimation. You should use scale factor of closer to 0.3/0.4 (so it looks closer to the white rectangle), so most of the depth pixels inside the rectangle will actually be the object, not the background/floor. Then it will take median depth value and use that as the Z.

Regarding the first image (jumping bounding box) - could you record some video of this? As boundingBox mappings can only be inside the actual detection.

kle

Hi erik ,

I understand, the scale factor is for fine tuning the spatial coordate when the blue box is inside the green box (green box is always around the detected object).
The scale factor doesn't expand when the blue box is long away from the green one (jumping).
I believe the scale factor is not the root cause of the jumping issue.

Could it be the logic issue of the depthai::node::YoloSpatialDetectionNetwork class?

I run 2 tests with 5 object, if the blue box is inside the green box then => OK.
The result is:

Test 1: Scale factor = 0.2 Result: 438 out of 744 detected objects are OK => Success rate: 58.9%
Test 2: Scale factor = 0.1 Result: 351 out of 744 detected objects are OK => Success rate: 47.2%

The result of spatial coordinates (x, z) only from the OK dectection above are

erik

Hi kle ,

The scale factor doesn't expand when the blue box is long away from the green one (jumping).

Do you have an example of that? BoundingBoxMappings should always be inside the detection itself.

kle

From the attached code above, it shows the outputs of the BoundingBoxMappings data and the detection data of each detected object, which returns from YoloSpatialDetectionNetwork (with scale factor = 1).
And the attached log outputs show the difference of those data happens from time to time.

YMIN and YMAX diff consistent. (Diff is within 1 => OK)
When it happens, it happened to ALL objects in that image/detection round.

Which means the BoundingBoxMappings is NOT always inside the detection itself in these data.
Or do I misunderstand something?