April tag object tracking working examples

MarilleTielens

Dear reader,

Over the past few weeks I have been working on a project but just can't seem to get it right. My goal is to track an object with an april tag (or a rig/cube of several of them for more accuracy) in a space of about 2x2x2m with one OAK -D Lite camera (using only the gray scale camera for less latency and more fps). The camera is hung about 220cm high on the wall at an angle looking down. In the this space I want to track the an object but relative to a point on the ground. So together with the depthai agents and claude code I have attempted to build a calibration script to properly define this new origin using 4four 17x17cm april tags in each corner of the space i want to track (all flush on the ground), where one of the outer corners functions as the origin. Aswell as a script to actively track the object which obviously depends on how good the calibration step is. I have gone through many iterations and I feel quite lost in the process. The detection works fine, but for the position estimation I have a lot of different errors and varrying results. I wonder if anyone has already built something similar I could use as a basis?

Oskar-Luxonis

Hey MarilleTielens - I don’t know of a drop-in example for this exact ground-frame setup. The v3 AprilTag example provides IDs and tag corners; pose must then be calculated on the host.

I’d calibrate once using all four floor tags’ known corner coordinates to obtain the camera-to-ground transform, then transform each object-tag pose into that frame. For a cube/rig, solve all visible tag corners together rather than averaging individual tag poses. Make sure the mono stream is undistorted and the intrinsics match the actual output resolution/crop—this is a common source of unstable results.

Note that on OAK-D Lite (RVC2), AprilTag detection runs on the host. You can also use our Luxonis skills with Codex or Claude to help build and debug the application.

Thanks,
Oskar