YoloSpatialDetectionNetwork bbox center only returning zero

sywise98

The image detection pipeline is not outputting a center point with is bounding box.

  "header": {
    "seq": 119,
    "stamp": {
      "sec": 1660853901,
      "nsec": 661912648
    },
    "frame_id": "front_camera_rgb_camera_optical_frame"
  },
  "detections": [
    {
      "results": [
        {
          "id": "0",
          "score": 0.6497554779052734
        }
      ],
      "bbox": {
        "center": {
          "x": 0,
          "y": 0,
          "theta": 0
        },
        "size_x": 52,
        "size_y": 142
      },
      "position": {
        "x": 1.1085253953933716,
        "y": 0.2005903124809265,
        "z": 3.2896978855133057
      },
      "is_tracking": false,
      "tracking_id": ""
    }
  ]
}`

luxonis-Sachin

Hello,
Can you provide more details on which version of ROS, depthai is used here ?
and can you cross check if you are using updated packages for this ?

sywise98

luxonis-Sachin

I'm using ros 1 noetic and the packages have been updated and upgrade

DanielCalaver

Hi,

I tried with OAK-D-S2 and OAK-D-Lite and worked for me on Ubuntu 20 and ROS1 Noetic, what device are you using? Are you running yolov4_publisher.launch from depthai-examples package?

results:
-
id: 56
score: 0.5338668823242188
bbox:
center:
x: 102.0
y: 227.5
theta: 0.0
size_x: 134.0
size_y: 135.0
position:
x: -0.24268843233585358
y: -0.045007672160863876
z: 1.2374526262283325
is_tracking: False
tracking_id: ''

sywise98

DanielCalaver

I'm using the oak-d camera on Ubuntu 20.04.4 LTS with ROS1 noetic. I made a custom node to run which gave me the zero but when I double checked my work by running the depthai_examples stereo_inertial_node and yolov4_publisher they also gave me zeros for some reason.

DanielCalaver

sywise98

Sorry for issues,

Did you installed the depthai-ros and depthai-ros-examples from the ROS binaries or compiled from source? If you compiled from source, did you used the latest branch on each?

What Calibration data is written on the device? You can check with something like this in a python script

import depthai as dai

pipeline = dai.Pipeline()

device = dai.Device()

device_calib = device.readCalibration()
eeprom_data = device_calib.eepromToJson()
print('eeprom_data = ')
print(eeprom_data)

sywise98

DanielCalaver

Its been installed by the from the ROS binaries and upated

    "boardName": "OAK-D",
    "boardRev": "R1M0E1",
    "cameraData": [
        [
            2,
            {
                "cameraType": 0,
                "distortionCoeff": [
                    -10.078534126281738,
                    97.58914947509766,
                    -0.000752690015360713,
                    0.0008525287266820669,
                    -63.12165451049805,
                    -10.12963581085205,
                    97.08373260498047,
                    -61.70897674560547,
                    0.0,
                    0.0,
                    0.0,
                    0.0,
                    0.0,
                    0.0
                ],
                "extrinsics": {
                    "rotationMatrix": [
                        [
                            0.9994920492172241,
                            0.03133673220872879,
                            -0.0057984525337815285
                        ],
                        [
                            -0.031284987926483154,
                            0.9994716644287109,
                            0.008809154853224754
                        ],
                        [
                            0.006071439478546381,
                            -0.00862327590584755,
                            0.9999443888664246
                        ]
                    ],
                    "specTranslation": {
                        "x": 3.75,
                        "y": 0.0,
                        "z": 0.0
                    },
                    "toCameraSocket": 0,
                    "translation": {
                        "x": 3.6940789222717285,
                        "y": 0.016222458332777023,
                        "z": -0.2880991995334625
                    }
                },
                "height": 800,
                "intrinsicMatrix": [
                    [
                        801.6409301757813,
                        0.0,
                        642.0193481445313
                    ],
                    [
                        0.0,
                        801.6409301757813,
                        394.6192932128906
                    ],
                    [
                        0.0,
                        0.0,
                        1.0
                    ]
                ],
                "lensPosition": 0,
                "specHfovDeg": 71.86000061035156,
                "width": 1280
            }
        ],
        [
            1,
            {
                "cameraType": 0,
                "distortionCoeff": [
                    -9.162176132202148,
                    93.17412567138672,
                    -0.0016400753520429134,
                    0.0007779067964293063,
                    -78.43282318115234,
                    -9.232532501220703,
                    92.77774810791016,
                    -77.2552490234375,
                    0.0,
                    0.0,
                    0.0,
                    0.0,
                    0.0,
                    0.0
                ],
                "extrinsics": {
                    "rotationMatrix": [
                        [
                            0.9998337030410767,
                            -0.017035847529768944,
                            0.006502930540591478
                        ],
                        [
                            0.017003335058689117,
                            0.9998428225517273,
                            0.005022567696869373
                        ],
                        [
                            -0.006587472278624773,
                            -0.0049111610278487206,
                            0.9999662637710571
                        ]
                    ],
                    "specTranslation": {
                        "x": -7.5,
                        "y": 0.0,
                        "z": 0.0
                    },
                    "toCameraSocket": 2,
                    "translation": {
                        "x": -7.474294662475586,
                        "y": -0.16496148705482483,
                        "z": 0.05168912932276726
                    }
                },
                "height": 800,
                "intrinsicMatrix": [
                    [
                        802.26611328125,
                        0.0,
                        639.803466796875
                    ],
                    [
                        0.0,
                        802.26611328125,
                        411.2508544921875
                    ],
                    [
                        0.0,
                        0.0,
                        1.0
                    ]
                ],
                "lensPosition": 0,
                "specHfovDeg": 71.86000061035156,
                "width": 1280
            }
        ],
        [
            0,
            {
                "cameraType": 0,
                "distortionCoeff": [
                    6.595644474029541,
                    -90.76404571533203,
                    -0.00031722753192298114,
                    0.0026943341363221407,
                    316.3438720703125,
                    6.388706684112549,
                    -89.15525817871094,
                    311.2258605957031,
                    0.0,
                    0.0,
                    0.0,
                    0.0,
                    0.0,
                    0.0
                ],
                "extrinsics": {
                    "rotationMatrix": [
                        [
                            0.0,
                            0.0,
                            0.0
                        ],
                        [
                            0.0,
                            0.0,
                            0.0
                        ],
                        [
                            0.0,
                            0.0,
                            0.0
                        ]
                    ],
                    "specTranslation": {
                        "x": -0.0,
                        "y": -0.0,
                        "z": -0.0
                    },
                    "toCameraSocket": -1,
                    "translation": {
                        "x": 0.0,
                        "y": 0.0,
                        "z": 0.0
                    }
                },
                "height": 2160,
                "intrinsicMatrix": [
                    [
                        3083.9609375,
                        0.0,
                        1924.1842041015625
                    ],
                    [
                        0.0,
                        3083.9609375,
                        1094.2318115234375
                    ],
                    [
                        0.0,
                        0.0,
                        1.0
                    ]
                ],
                "lensPosition": 124,
                "specHfovDeg": 68.7938003540039,
                "width": 3840
            }
        ]
    ],
    "imuExtrinsics": {
        "rotationMatrix": [
            [
                0.0,
                0.0,
                0.0
            ],
            [
                0.0,
                0.0,
                0.0
            ],
            [
                0.0,
                0.0,
                0.0
            ]
        ],
        "specTranslation": {
            "x": 0.0,
            "y": 0.0,
            "z": 0.0
        },
        "toCameraSocket": -1,
        "translation": {
            "x": 0.0,
            "y": 0.0,
            "z": 0.0
        }
    },
    "miscellaneousData": [],
    "stereoRectificationData": {
        "leftCameraSocket": 1,
        "rectifiedRotationLeft": [
            [
                0.9999871253967285,
                0.005063802003860474,
                -0.0003014907124452293
            ],
            [
                -0.005063039716333151,
                0.999984085559845,
                0.0024773816112428904
            ],
            [
                0.00031403088360093534,
                -0.002475823275744915,
                0.9999969005584717
            ]
        ],
        "rectifiedRotationRight": [
            [
                0.9997326135635376,
                0.022064609453082085,
                -0.006913736928254366
            ],
            [
                -0.022081665694713593,
                0.9997532963752747,
                -0.0024004322476685047
            ],
            [
                0.006859066896140575,
                0.0025524573866277933,
                0.9999732375144958
            ]
        ],
        "rightCameraSocket": 2
    },
    "version": 6
}`

luxonis-Sachin

sywise98 What version of ROS binaries do you have as of now ?
The issue was fixed on 2.5.3 release here. it should be available in a week or two if you want to just install from debs.

DanielCalaver

I see the same bug with OAK-D and packages from ROS binaries, until we fix this @sywise98 could you try and build from sources? Instructions here https://github.com/luxonis/depthai-ros

cc @luxonis-Sachin

sywise98

luxonis-Sachin

I'm on depthai-ros 2.5.2 I'll update and see what happens.

Edit:
I've used the update and it working. Thank you