YoloSpatialDetectionNetwork bbox center only returning zero

Ssywise98 · Aug 18, 2022

The image detection pipeline is not outputting a center point with is bounding box.

  "header": {
    "seq": 119,
    "stamp": {
      "sec": 1660853901,
      "nsec": 661912648
    },
    "frame_id": "front_camera_rgb_camera_optical_frame"
  },
  "detections": [
    {
      "results": [
        {
          "id": "0",
          "score": 0.6497554779052734
        }
      ],
      "bbox": {
        "center": {
          "x": 0,
          "y": 0,
          "theta": 0
        },
        "size_x": 52,
        "size_y": 142
      },
      "position": {
        "x": 1.1085253953933716,
        "y": 0.2005903124809265,
        "z": 3.2896978855133057
      },
      "is_tracking": false,
      "tracking_id": ""
    }
  ]
}`

Lluxonis-Sachin · Aug 18, 2022

Hello,
Can you provide more details on which version of ROS, depthai is used here ?
and can you cross check if you are using updated packages for this ?

Ssywise98 · Aug 19, 2022

luxonis-Sachin

I'm using ros 1 noetic and the packages have been updated and upgrade

DDanielCalaver · Aug 22, 2022

Hi,

I tried with OAK-D-S2 and OAK-D-Lite and worked for me on Ubuntu 20 and ROS1 Noetic, what device are you using? Are you running yolov4_publisher.launch from depthai-examples package?

results:
-
id: 56
score: 0.5338668823242188
bbox:
center:
x: 102.0
y: 227.5
theta: 0.0
size_x: 134.0
size_y: 135.0
position:
x: -0.24268843233585358
y: -0.045007672160863876
z: 1.2374526262283325
is_tracking: False
tracking_id: ''

Ssywise98 · Aug 22, 2022

DanielCalaver

I'm using the oak-d camera on Ubuntu 20.04.4 LTS with ROS1 noetic. I made a custom node to run which gave me the zero but when I double checked my work by running the depthai_examples stereo_inertial_node and yolov4_publisher they also gave me zeros for some reason.

DDanielCalaver · Aug 22, 2022

sywise98

Sorry for issues,

Did you installed the depthai-ros and depthai-ros-examples from the ROS binaries or compiled from source? If you compiled from source, did you used the latest branch on each?

What Calibration data is written on the device? You can check with something like this in a python script

import depthai as dai

pipeline = dai.Pipeline()

device = dai.Device()

device_calib = device.readCalibration()
eeprom_data = device_calib.eepromToJson()
print('eeprom_data = ')
print(eeprom_data)

Ssywise98 · Aug 22, 2022

DanielCalaver

Its been installed by the from the ROS binaries and upated

    "boardName": "OAK-D",
    "boardRev": "R1M0E1",
    "cameraData": [
        [
            2,
            {
                "cameraType": 0,
                "distortionCoeff": [
                    -10.078534126281738,
                    97.58914947509766,
                    -0.000752690015360713,
                    0.0008525287266820669,
                    -63.12165451049805,
                    -10.12963581085205,
                    97.08373260498047,
                    -61.70897674560547,
                    0.0,
                    0.0,
                    0.0,
                    0.0,
                    0.0,
                    0.0
                ],
                "extrinsics": {
                    "rotationMatrix": [
                        [
                            0.9994920492172241,
                            0.03133673220872879,
                            -0.0057984525337815285
                        ],
                        [
                            -0.031284987926483154,
                            0.9994716644287109,
                            0.008809154853224754
                        ],
                        [
                            0.006071439478546381,
                            -0.00862327590584755,
                            0.9999443888664246
                        ]
                    ],
                    "specTranslation": {
                        "x": 3.75,
                        "y": 0.0,
                        "z": 0.0
                    },
                    "toCameraSocket": 0,
                    "translation": {
                        "x": 3.6940789222717285,
                        "y": 0.016222458332777023,
                        "z": -0.2880991995334625
                    }
                },
                "height": 800,
                "intrinsicMatrix": [
                    [
                        801.6409301757813,
                        0.0,
                        642.0193481445313
                    ],
                    [
                        0.0,
                        801.6409301757813,
                        394.6192932128906
                    ],
                    [
                        0.0,
                        0.0,
                        1.0
                    ]
                ],
                "lensPosition": 0,
                "specHfovDeg": 71.86000061035156,
                "width": 1280
            }
        ],
        [
            1,
            {
                "cameraType": 0,
                "distortionCoeff": [
                    -9.162176132202148,
                    93.17412567138672,
                    -0.0016400753520429134,
                    0.0007779067964293063,
                    -78.43282318115234,
                    -9.232532501220703,
                    92.77774810791016,
                    -77.2552490234375,
                    0.0,
                    0.0,
                    0.0,
                    0.0,
                    0.0,
                    0.0
                ],
                "extrinsics": {
                    "rotationMatrix": [
                        [
                            0.9998337030410767,
                            -0.017035847529768944,
                            0.006502930540591478
                        ],
                        [
                            0.017003335058689117,
                            0.9998428225517273,
                            0.005022567696869373
                        ],
                        [
                            -0.006587472278624773,
                            -0.0049111610278487206,
                            0.9999662637710571
                        ]
                    ],
                    "specTranslation": {
                        "x": -7.5,
                        "y": 0.0,
                        "z": 0.0
                    },
                    "toCameraSocket": 2,
                    "translation": {
                        "x": -7.474294662475586,
                        "y": -0.16496148705482483,
                        "z": 0.05168912932276726
                    }
                },
                "height": 800,
                "intrinsicMatrix": [
                    [
                        802.26611328125,
                        0.0,
                        639.803466796875
                    ],
                    [
                        0.0,
                        802.26611328125,
                        411.2508544921875
                    ],
                    [
                        0.0,
                        0.0,
                        1.0
                    ]
                ],
                "lensPosition": 0,
                "specHfovDeg": 71.86000061035156,
                "width": 1280
            }
        ],
        [
            0,
            {
                "cameraType": 0,
                "distortionCoeff": [
                    6.595644474029541,
                    -90.76404571533203,
                    -0.00031722753192298114,
                    0.0026943341363221407,
                    316.3438720703125,
                    6.388706684112549,
                    -89.15525817871094,
                    311.2258605957031,
                    0.0,
                    0.0,
                    0.0,
                    0.0,
                    0.0,
                    0.0
                ],
                "extrinsics": {
                    "rotationMatrix": [
                        [
                            0.0,
                            0.0,
                            0.0
                        ],
                        [
                            0.0,
                            0.0,
                            0.0
                        ],
                        [
                            0.0,
                            0.0,
                            0.0
                        ]
                    ],
                    "specTranslation": {
                        "x": -0.0,
                        "y": -0.0,
                        "z": -0.0
                    },
                    "toCameraSocket": -1,
                    "translation": {
                        "x": 0.0,
                        "y": 0.0,
                        "z": 0.0
                    }
                },
                "height": 2160,
                "intrinsicMatrix": [
                    [
                        3083.9609375,
                        0.0,
                        1924.1842041015625
                    ],
                    [
                        0.0,
                        3083.9609375,
                        1094.2318115234375
                    ],
                    [
                        0.0,
                        0.0,
                        1.0
                    ]
                ],
                "lensPosition": 124,
                "specHfovDeg": 68.7938003540039,
                "width": 3840
            }
        ]
    ],
    "imuExtrinsics": {
        "rotationMatrix": [
            [
                0.0,
                0.0,
                0.0
            ],
            [
                0.0,
                0.0,
                0.0
            ],
            [
                0.0,
                0.0,
                0.0
            ]
        ],
        "specTranslation": {
            "x": 0.0,
            "y": 0.0,
            "z": 0.0
        },
        "toCameraSocket": -1,
        "translation": {
            "x": 0.0,
            "y": 0.0,
            "z": 0.0
        }
    },
    "miscellaneousData": [],
    "stereoRectificationData": {
        "leftCameraSocket": 1,
        "rectifiedRotationLeft": [
            [
                0.9999871253967285,
                0.005063802003860474,
                -0.0003014907124452293
            ],
            [
                -0.005063039716333151,
                0.999984085559845,
                0.0024773816112428904
            ],
            [
                0.00031403088360093534,
                -0.002475823275744915,
                0.9999969005584717
            ]
        ],
        "rectifiedRotationRight": [
            [
                0.9997326135635376,
                0.022064609453082085,
                -0.006913736928254366
            ],
            [
                -0.022081665694713593,
                0.9997532963752747,
                -0.0024004322476685047
            ],
            [
                0.006859066896140575,
                0.0025524573866277933,
                0.9999732375144958
            ]
        ],
        "rightCameraSocket": 2
    },
    "version": 6
}`

DDanielCalaver · Aug 23, 2022

I see the same bug with OAK-D and packages from ROS binaries, until we fix this @sywise98 could you try and build from sources? Instructions here https://github.com/luxonis/depthai-ros

cc @luxonis-Sachin

Lluxonis-Sachin · Aug 23, 2022

sywise98 What version of ROS binaries do you have as of now ?
The issue was fixed on 2.5.3 release here. it should be available in a week or two if you want to just install from debs.

Ssywise98 · Aug 23, 2022

luxonis-Sachin

I'm on depthai-ros 2.5.2 I'll update and see what happens.

Edit:
I've used the update and it working. Thank you