Transforming OAK D Lite frame to Panda_arm frame

JideJimoh

Hello, I am trying to do a pick and place with franka panda arm and oak d lite,
With the oak I have combined the object tracking and spatial coordinate example to detect and get the coordinate of the object detected and sending that to the robot for movement but it's not working. i need to transform the panda frame to camera frame, but I can find the frame details of the camera or maybe I just don't know( I am new to this)

Here's an example code of what I'm trying to achieve

In the code they have

get_transform("panda_link0", "panda_hand_tcp")

I need to replace panda_hand_tcp with oak-d-lite frame

Thank you

erik

Hi @JideJimoh ,
So OAK-D-Lite is on the arm itself? So you need to combine extrinsics from the arm, add the extrinsics from camera mounting to camera frame (middle camera), and then add spatial coordinates of the objects OAK-D-Lite detects (XYZ in mm), so you can get object to arm extrinsics?

JideJimoh

So i found depthai-ros which provide this value and able to transform between both frame, the question I have now is how can I get orientation from the camera for transformation as well, the arm is moving in some weird way and I think not setting orientation could be the problem.

jakaskerl

Hi @JideJimoh
I'd suggest you first calibrate the camera to the world coordinates using a chessboard pattern: luxonis/depthai-experimentstree/master/gen2-multiple-devices/multi-cam-calibration.

The script will give you position of the oak-d relative to chessboard, which you can use to compute the world to camera transformation. Then combine with the world to arm to get the full transform which you can then use even when the arm is moving.

Thanks,
Jaka