Environment
depthai 3.7.1 (Python)
OAK-D-W-POE
bootloader 0.0.28
Summary
Setting a PointCloud node's target coordinate system to a HousingCoordinateSystem (e.g. CAM_A) re-orients the cloud but does not translate it — the camera→housing translation is omitted. The same target given as a CameraBoardSocket (CAM_A) applies the full rigid transform (rotation and translation), and getHousingCalibration(CAM_B, CAM_A) itself returns a matrix that includes that translation. So the node's housing-target output disagrees with depthai's own calibration getter.
What the calibration reports (both agree — CAM_A sits ≈ half the 7.5 cm baseline from the left mono):
getCameraExtrinsics(CAM_B -> CAM_A) translation (m): [-0.0376012 -0.0004297 -0.0000269]
getHousingCalibration(CAM_B -> CAM_A) translation (m): [-0.0375013 -0.0009306 -0.0000119]
What the PointCloud node actually applies (fit between a plain-pinhole deprojection of the same depth frame and each target's organized output):
target = CameraBoardSocket.CAM_A applied translation (m): [-0.037601 -0.000416 -0.000303] # ✓ matches getCameraExtrinsics
target = HousingCoordinateSystem.CAM_A applied translation (m): [ 0.0000011 -0.0000040 -0.000169] # ✗ translation ~0 (rotation only)
Expected: the housing target applies the translation getHousingCalibration reports (≈ −0.0375 m in x), consistent with the CameraBoardSocket target. Per the docs, housing CAM_A is "positioned at camera sensor spec locations," i.e. it has a defined origin, not just an orientation.
Actual: the housing target applies only the rotation; the cloud origin stays at the depth camera.
MRE (standalone, depthai + numpy; pass the device IP as the first arg for PoE):
#!/usr/bin/env python3
"""MRE: PointCloud HousingCoordinateSystem target omits the extrinsic translation.
For a CameraBoardSocket target the PointCloud node applies the full camera->target
rigid transform (translation matches getCameraExtrinsics). For a
HousingCoordinateSystem target it applies only the rotation -- the translation
that getHousingCalibration reports for the same frame is dropped.
We deproject the depth frame ourselves with a plain pinhole, then fit the rigid
transform between that and each target's organized output; the fitted translation
is exactly what that node applied. Both targets run as separate PointCloud nodes
in one pipeline (reconnecting or building a second pipeline both fail on RVC2 /
depthai 3.7.1), each with its own passthroughDepth so cloud<->depth stay paired.
Usage: python mre.py [device_ip]
"""
from __future__ import annotations
import sys
import depthai as dai
import numpy as np
SIZE = (640, 400)
FRAMES = 15
TARGETS = {
'CameraBoardSocket.CAM_A': dai.CameraBoardSocket.CAM_A,
'HousingCoordinateSystem.CAM_A': dai.HousingCoordinateSystem.CAM_A,
}
def kabsch(src, dst):
src_c, dst_c = src.mean(axis=0), dst.mean(axis=0)
h = (src - src_c).T @ (dst - dst_c)
u, _, vt = np.linalg.svd(h)
d = np.sign(np.linalg.det(vt.T @ u.T))
r = vt.T @ np.diag([1.0, 1.0, d]) @ u.T
return r, dst_c - r @ src_c
def measure(device, k):
pipeline = dai.Pipeline(device)
cam_l = pipeline.create(dai.node.Camera).build(dai.CameraBoardSocket.CAM_B)
cam_r = pipeline.create(dai.node.Camera).build(dai.CameraBoardSocket.CAM_C)
stereo = pipeline.create(dai.node.StereoDepth)
cam_l.requestOutput(SIZE).link(stereo.left)
cam_r.requestOutput(SIZE).link(stereo.right)
cloud_qs, depth_qs = {}, {}
for name, target in TARGETS.items():
pc = pipeline.create(dai.node.PointCloud)
pc.initialConfig.setLengthUnit(dai.LengthUnit.METER)
pc.initialConfig.setOrganized(True) # W*H, pixel order
pc.setTargetCoordinateSystem(target)
stereo.depth.link(pc.inputDepth)
cloud_qs[name] = pc.outputPointCloud.createOutputQueue(maxSize=4, blocking=False)
depth_qs[name] = pc.passthroughDepth.createOutputQueue(maxSize=4, blocking=False)
with pipeline:
pipeline.start()
clouds, depths = {}, {}
for _ in range(FRAMES):
for name in TARGETS:
clouds[name] = cloud_qs[name].get()
depths[name] = depth_qs[name].get()
results = {}
for name in TARGETS:
depth, cloud = depths[name], clouds[name]
w, h = depth.getWidth(), depth.getHeight()
d = np.frombuffer(depth.getData(), dtype=np.uint16).reshape(h, w).astype(np.float32) / 1000.0
us, vs = np.meshgrid(np.arange(w), np.arange(h))
ours = np.stack(
[(us - k[0, 2]) * d / k[0, 0], (vs - k[1, 2]) * d / k[1, 1], d], axis=-1
).reshape(-1, 3)
theirs = np.asarray(cloud.getPoints()).reshape(-1, 3)
mask = (d.reshape(-1) > 0) & np.isfinite(theirs).all(axis=1)
results[name] = kabsch(ours[mask], theirs[mask])
return results
def main():
ip = sys.argv[1] if len(sys.argv) > 1 else ''
device = dai.Device(dai.DeviceInfo(ip)) if ip else dai.Device()
with device:
calib = device.readCalibration()
left = calib.getStereoLeftCameraId()
k = np.array(calib.getCameraIntrinsics(left, *SIZE))
print(f'depthai {dai.__version__} | {device.getPlatformAsString()} {device.getDeviceName()}')
ext_t = np.array(calib.getCameraExtrinsics(
left, dai.CameraBoardSocket.CAM_A, unit=dai.LengthUnit.METER))[:3, 3]
hou_t = np.array(calib.getHousingCalibration(
left, dai.HousingCoordinateSystem.CAM_A, unit=dai.LengthUnit.METER))[:3, 3]
print(f'\ngetCameraExtrinsics(CAM_B -> CAM_A) translation (m): {ext_t}')
print(f'getHousingCalibration(CAM_B -> CAM_A) translation (m): {hou_t}')
results = measure(device, k)
for name, (_, t) in results.items():
print(f'\n--- target = {name} ---')
print(f' translation the node ACTUALLY applied (m): {t}')
if __name__ == '__main__':
main()
Output on our device:
depthai 3.7.1 | RVC2 OAK-D-W-POE
getCameraExtrinsics(CAM_B -> CAM_A) translation (m): [-3.76e-02 -4.30e-04 -2.69e-05]
getHousingCalibration(CAM_B -> CAM_A) translation (m): [-3.75e-02 -9.31e-04 -1.19e-05]
--- target = CameraBoardSocket.CAM_A ---
translation the node ACTUALLY applied (m): [-0.037601 -0.000416 -0.000303]
--- target = HousingCoordinateSystem.CAM_A ---
translation the node ACTUALLY applied (m): [ 0.0000011 -0.0000040 -0.000169]
