Spatial Object Tracker API Demo Code - Oak Wide vs Pro

DDavidOS · Aug 14, 2023

Hi,

I have been working on the API example code for the RGB Spatial Object Tracker here:

https://docs.luxonis.com/projects/api/en/latest/samples/ObjectTracker/spatial_object_tracker/

If I use my Oak D Pro (https://docs.luxonis.com/projects/hardware/en/latest/pages/DM9098pro/) it works correctly with no issues.

If I try to run exactly the same code on my Oak D Wide (https://docs.luxonis.com/projects/hardware/en/latest/pages/DM9098w/) it does not work.

The Preview screen which you see with the Pro never appears on my desktop (Windows 11) when using the Wide. I can see that the console executes Line 95 print(f"Show Preview: ") but looking at the debug the preview variable on Line 96 never seems to get assigned as device.getOutputQueue("preview", 4, False) never returns a value. There are no error mesages, and no time outs the code just sits there until ctrl-c is used to interrupt the execution.

Any ideas why this is happening please?

Thanks,

David

jakaskerl · Aug 14, 2023

Hi DavidOS
Could you try running the script with pipeline graph tool to see where the blockage occurs?
Do simpler examples work as expected (rgb preview, cam_test, etc)?

Thanks,
Jaka

DDavidOS · Aug 15, 2023

Hi jakaskerl

I have just run some analysis and here are the reults:

RGB Preview works on both cameras:

Connected cameras: [{socket: CAM_A, sensorName: IMX378, width: 4056, height: 3040, orientation: AUTO, supportedTypes: [COLOR], hasAutofocus: 1, name: color}, {socket: CAM_B, sensorName: OV9282, width: 1280, height: 800, orientation: AUTO, supportedTypes: [MONO], hasAutofocus: 0, name: left}, {socket: CAM_C, sensorName: OV9282, width: 1280, height: 800, orientation: AUTO, supportedTypes: [MONO], hasAutofocus: 0, name: right}]

Usb speed: SUPER

Device name: OAK-D-W

Connected cameras: [{socket: CAM_A, sensorName: IMX378, width: 4056, height: 3040, orientation: AUTO, supportedTypes: [COLOR], hasAutofocus: 1, name: color}, {socket: CAM_B, sensorName: OV9282, width: 1280, height: 800, orientation: AUTO, supportedTypes: [MONO], hasAutofocus: 0, name: left}, {socket: CAM_C, sensorName: OV9282, width: 1280, height: 800, orientation: AUTO, supportedTypes: [MONO], hasAutofocus: 0, name: right}]

Usb speed: SUPER

Device name: OAK-D-PRO-AF

Attached are the two Pipeline Graphs.

Please let me know if there is any other information you need.

Thanks again,

David

jakaskerl · Aug 16, 2023

Hi DavidOS
Everything looks as per usual here, I guess you will have to try and figure out which part of the pipeline causes the problems. I suggest going from left to right and viewing the outputs of the last node in the chain. Eg, first remove anything that isn't color, mono or depth node. This way you will see if it's the depth that is causing problems. Then add spatialdetectionnetwork and do the same. Very likely it has something to do with the calibration since its device specific.

Thanks
Jaka

DDavidOS · Aug 29, 2023

Hi, I have been revisiting this today and found that the 'left' mono camera on the Oak D Wide seems to produce no output so when I run the guidied-manual-tests.py script in the depthai repo none of the depth tests work. The Oak D Pro runs all the guided tests correctly.

Is there a way to factory reset the calibration in the Oak D Wide?

I have looked in the API docs but I am unsure how to factory reset a camera rather than run the whole chequer board calibration routine.

Thanks for you help,

David

jakaskerl · Aug 30, 2023

Hi DavidOS
The left mono camera provides no frames even when not part of the stereo? Do standard mono examples and/or cam_test.py work as expected?

Thanks,
Jaka

DDavidOS · Aug 30, 2023

Hi jakaskerl

I have just been trying this again - left camera was working on the examples but now it is not. However, if I run the examples/Camera/camera_isp.py script the Left camera reports it's details correctly.

If I run the mono_preview.py script I now get the following:

[18443010F1C4A90F00] [20.1] [4.747] [system] [critical] Fatal error. Please report to developers. Log: 'Fatal error on MSS CPU: trap: 09, address: 8022FDC4' '0'

Traceback (most recent call last):

File "depthai-python/examples/MonoCamera/mono_preview.py", line 37, in <module>

inLeft = qLeft.tryGet()

^^^^^^^^^^^^^^

RuntimeError: Communication exception - possible device error/misconfiguration. Original message 'Couldn't read data from stream: 'left' (X_LINK_ERROR)'

Where as before (i.e. when I last played with this 2wk ago) this worked ok.

Thanks again for your help,

David

jakaskerl · Aug 30, 2023

Hi DavidOS
Could you try cam_test.py with the debug enabled? Please paste the logs below.

EDIT:
Also; have any changes been made to the device -hardware-wise:

different usb cable
different powering solution
etc

Thanks,
Jaka

DDavidOS · Aug 30, 2023

Hi jakaskerl

Here is the debug - FWIW I also ran the debug on the Pro and it worked fine but failed on the Wide. Ther logs looked pretty much the same bar time stamps and unique IDs

Same USB cable - one out ouf the Luxonis Box

Same USB Power - Standard USB3 from a PC port.

I have tried on RPi 4 with Bullseye, Win 11 and macOS Monterey. All have exactly the same symptoms. In all cases the Left Cam does not work on the WIDE and the Pro works fine. It seems to me there could be a hardware issue with the Wide

Thanks again for your help,

David.

EDIT: Forgot to say the Pro log gave FPS readings of ~30 across all 3 cameras rather than zero seen below)

[2023-08-30 21:21:42.729] [depthai] [debug] Python bindings - version: 2.22.0.0 from 2023-06-13 02:13:35 +0200 build: 2023-06-13 01:26:45 +0000

[2023-08-30 21:21:42.729] [depthai] [debug] Library information - version: 2.22.0, commit: 82ab07d037c02f56042d1d2d55a718f379651ed9 from 2023-06-13 02:13:14 +0200, build: 2023-06-13 01:26:33 +0000

[2023-08-30 21:21:42.731] [depthai] [debug] Initialize - finished

[2023-08-30 21:21:42.786] [depthai] [debug] Resources - Archive 'depthai-bootloader-fwp-0.0.26.tar.xz' open: 1ms, archive read: 55ms

[2023-08-30 21:21:43.038] [depthai] [debug] Resources - Archive 'depthai-device-fwp-f033fd9c7eb0b3578d12f90302e87759c78cfb36.tar.xz' open: 1ms, archive read: 307ms

[2023-08-30 21:22:07.869] [depthai] [debug] Python bindings - version: 2.22.0.0 from 2023-06-13 02:13:35 +0200 build: 2023-06-13 01:26:45 +0000

[2023-08-30 21:22:07.869] [depthai] [debug] Library information - version: 2.22.0, commit: 82ab07d037c02f56042d1d2d55a718f379651ed9 from 2023-06-13 02:13:14 +0200, build: 2023-06-13 01:26:33 +0000

[2023-08-30 21:22:07.871] [depthai] [debug] Initialize - finished

Enabled cameras:

rgb : color

left : mono

right : mono

DepthAI version: 2.22.0.0

DepthAI path: XX\Packages\PythonSoftwareFoundation.Python.3.11_qbz5n2kfra8p0\LocalCache\local-packages\Python311\site-packages\depthai.cp311-win_amd64.pyd

c:\code_samples\depthai-python\utilities\cam_test.py:284: DeprecationWarning: Use constructor taking 'UsbSpeed' instead

with dai.Device(*dai_device_args) as device:

[2023-08-30 21:22:07.927] [depthai] [debug] Resources - Archive 'depthai-bootloader-fwp-0.0.26.tar.xz' open: 1ms, archive read: 56ms

[2023-08-30 21:22:08.173] [depthai] [debug] Resources - Archive 'depthai-device-fwp-f033fd9c7eb0b3578d12f90302e87759c78cfb36.tar.xz' open: 1ms, archive read: 302ms

[2023-08-30 21:22:09.144] [depthai] [debug] Searching for booted device: DeviceInfo(name=1.5, mxid=18443010F1C4A90F00, X_LINK_BOOTED, X_LINK_USB_VSC, X_LINK_MYRIAD_X, X_LINK_SUCCESS), name used as hint only

[18443010F1C4A90F00] [1.5] [0.622] [system] [info] Memory Usage - DDR: 0.12 / 337.18 MiB, CMX: 2.04 / 2.50 MiB, LeonOS Heap: 7.50 / 80.03 MiB, LeonRT Heap: 2.89 / 41.06 MiB

[18443010F1C4A90F00] [1.5] [0.622] [system] [info] Temperatures - Average: 29.10C, CSS: 30.08C, MSS 28.62C, UPA: 28.62C, DSS: 29.10C

[18443010F1C4A90F00] [1.5] [0.622] [system] [info] Cpu Usage - LeonOS 79.37%, LeonRT: 2.10%

[18443010F1C4A90F00] [1.5] [1693426929.785] [host] [debug] Schema dump: {"connections":[{"node1Id":3,"node1Output":"isp","node1OutputGroup":"","node2Id":2,"node2Input":"in","node2InputGroup":""},{"node1Id":0,"node1Output":"out","node1OutputGroup":"","node2Id":3,"node2Input":"inputControl","node2InputGroup":""},{"node1Id":5,"node1Output":"out","node1OutputGroup":"","node2Id":4,"node2Input":"in","node2InputGroup":""},{"node1Id":0,"node1Output":"out","node1OutputGroup":"","node2Id":5,"node2Input":"inputControl","node2InputGroup":""},{"node1Id":7,"node1Output":"out","node1OutputGroup":"","node2Id":6,"node2Input":"in","node2InputGroup":""},{"node1Id":0,"node1Output":"out","node1OutputGroup":"","node2Id":7,"node2Input":"inputControl","node2InputGroup":""}],"globalProperties":{"calibData":null,"cameraTuningBlobSize":null,"cameraTuningBlobUri":"","leonCssFrequencyHz":700000000.0,"leonMssFrequencyHz":700000000.0,"pipelineName":null,"pipelineVersion":null,"sippBufferSize":18432,"sippDmaBufferSize":16384,"xlinkChunkSize":-1},"nodes":[[0,{"id":0,"ioInfo":[[["","out"],{"blocking":false,"group":"","id":1,"name":"out","queueSize":8,"type":0,"waitForMessage":false}]],"name":"XLinkIn","properties":[185,3,189,7,99,111,110,116,114,111,108,130,0,0,80,0,8]}],[1,{"id":1,"ioInfo":[[["","out"],{"blocking":false,"group":"","id":2,"name":"out","queueSize":8,"type":0,"waitForMessage":false}]],"name":"XLinkIn","properties":[185,3,189,9,116,111,102,67,111,110,102,105,103,130,0,0,80,0,8]}],[2,{"id":2,"ioInfo":[[["","in"],{"blocking":true,"group":"","id":3,"name":"in","queueSize":8,"type":3,"waitForMessage":true}]],"name":"XLinkOut","properties":[185,3,136,0,0,128,191,189,3,114,103,98,0]}],[3,{"id":3,"ioInfo":[[["","inputConfig"],{"blocking":false,"group":"","id":4,"name":"inputConfig","queueSize":8,"type":3,"waitForMessage":false}],[["","raw"],{"blocking":false,"group":"","id":9,"name":"raw","queueSize":8,"type":0,"waitForMessage":false}],[["","still"],{"blocking":false,"group":"","id":10,"name":"still","queueSize":8,"type":0,"waitForMessage":false}],[["","inputControl"],{"blocking":true,"group":"","id":5,"name":"inputControl","queueSize":8,"type":3,"waitForMessage":false}],[["","video"],{"blocking":false,"group":"","id":6,"name":"video","queueSize":8,"type":0,"waitForMessage":false}],[["","isp"],{"blocking":false,"group":"","id":7,"name":"isp","queueSize":8,"type":0,"waitForMessage":false}],[["","preview"],{"blocking":false,"group":"","id":8,"name":"preview","queueSize":8,"type":0,"waitForMessage":false}],[["","frameEvent"],{"blocking":false,"group":"","id":11,"name":"frameEvent","queueSize":8,"type":0,"waitForMessage":false}]],"name":"ColorCamera","properties":[185,26,185,27,0,3,0,0,0,185,3,0,0,0,185,5,0,0,0,0,0,185,5,0,0,0,0,0,0,0,0,0,0,0,0,185,3,0,0,0,185,3,0,0,0,0,0,0,0,0,0,0,0,0,0,0,189,0,255,0,1,0,129,44,1,129,44,1,255,255,255,255,0,136,0,0,240,65,0,136,0,0,128,191,136,0,0,128,191,1,185,4,1,1,1,1,3,3,4,4,4,190]}],[4,{"id":4,"ioInfo":[[["","in"],{"blocking":true,"group":"","id":12,"name":"in","queueSize":8,"type":3,"waitForMessage":true}]],"name":"XLinkOut","properties":[185,3,136,0,0,128,191,189,4,108,101,102,116,0]}],[5,{"id":5,"ioInfo":[[["","inputControl"],{"blocking":true,"group":"","id":13,"name":"inputControl","queueSize":8,"type":3,"waitForMessage":false}],[["","out"],{"blocking":false,"group":"","id":14,"name":"out","queueSize":8,"type":0,"waitForMessage":false}],[["","raw"],{"blocking":false,"group":"","id":15,"name":"raw","queueSize":8,"type":0,"waitForMessage":false}],[["","frameEvent"],{"blocking":false,"group":"","id":16,"name":"frameEvent","queueSize":8,"type":0,"waitForMessage":false}]],"name":"MonoCamera","properties":[185,10,185,27,0,3,0,0,0,185,3,0,0,0,185,5,0,0,0,0,0,185,5,0,0,0,0,0,0,0,0,0,0,0,0,185,3,0,0,0,185,3,0,0,0,0,0,0,0,0,0,0,0,0,0,1,189,0,255,1,136,0,0,240,65,0,3,3,190]}],[6,{"id":6,"ioInfo":[[["","in"],{"blocking":true,"group":"","id":17,"name":"in","queueSize":8,"type":3,"waitForMessage":true}]],"name":"XLinkOut","properties":[185,3,136,0,0,128,191,189,5,114,105,103,104,116,0]}],[7,{"id":7,"ioInfo":[[["","inputControl"],{"blocking":true,"group":"","id":18,"name":"inputControl","queueSize":8,"type":3,"waitForMessage":false}],[["","out"],{"blocking":false,"group":"","id":19,"name":"out","queueSize":8,"type":0,"waitForMessage":false}],[["","raw"],{"blocking":false,"group":"","id":20,"name":"raw","queueSize":8,"type":0,"waitForMessage":false}],[["","frameEvent"],{"blocking":false,"group":"","id":21,"name":"frameEvent","queueSize":8,"type":0,"waitForMessage":false}]],"name":"MonoCamera","properties":[185,10,185,27,0,3,0,0,0,185,3,0,0,0,185,5,0,0,0,0,0,185,5,0,0,0,0,0,0,0,0,0,0,0,0,185,3,0,0,0,185,3,0,0,0,0,0,0,0,0,0,0,0,0,0,2,189,0,255,1,136,0,0,240,65,0,3,3,190]}]]}

[18443010F1C4A90F00] [1.5] [1693426929.786] [host] [debug] Asset map dump: {"map":{}}

[18443010F1C4A90F00] [1.5] [0.663] [MonoCamera(7)] [info] Using board socket: 2, id: 2

[18443010F1C4A90F00] [1.5] [0.663] [MonoCamera(5)] [info] Using board socket: 1, id: 1

[18443010F1C4A90F00] [1.5] [0.665] [system] [info] SIPP (Signal Image Processing Pipeline) internal buffer size '18432'B, DMA buffer size: '16384'B

Connected cameras:

[18443010F1C4A90F00] [1.5] [0.700] [system] [info] ColorCamera allocated resources: no shaves; cmx slices: [13-15]

MonoCamera allocated resources: no shaves; cmx slices: [13-15]

-socket CAM_A : IMX378 4056 x 3040 focus:auto - COLOR

-socket CAM_B : OV9282 1280 x 800 focus:fixed - MONO

-socket CAM_C : OV9282 1280 x 800 focus:fixed - MONO

USB speed: SUPER

IR drivers: []

Cam: rgb left right [host | capture timestamp]

FPS: 0.00| 0.00 0.00| 0.00 0.00| 0.00 [18443010F1C4A90F00] [1.5] [1.623] [system] [info] Memory Usage - DDR: 109.69 / 337.18 MiB, CMX: 2.10 / 2.50 MiB, LeonOS Heap: 48.33 / 80.03 MiB, LeonRT Heap: 4.19 / 41.06 MiB

[18443010F1C4A90F00] [1.5] [1.623] [system] [info] Temperatures - Average: 30.50C, CSS: 31.77C, MSS 30.08C, UPA: 29.59C, DSS: 30.56C

[18443010F1C4A90F00] [1.5] [1.623] [system] [info] Cpu Usage - LeonOS 23.88%, LeonRT: 7.89%

FPS: 0.00| 0.00 0.00| 0.00 0.00| 0.00 [18443010F1C4A90F00] [1.5] [2.650] [system] [info] Memory Usage - DDR: 109.69 / 337.18 MiB, CMX: 2.10 / 2.50 MiB, LeonOS Heap: 48.33 / 80.03 MiB, LeonRT Heap: 4.19 / 41.06 MiB

[18443010F1C4A90F00] [1.5] [2.650] [system] [info] Temperatures - Average: 30.86C, CSS: 32.25C, MSS 30.32C, UPA: 30.32C, DSS: 30.56C

[18443010F1C4A90F00] [1.5] [2.650] [system] [info] Cpu Usage - LeonOS 16.10%, LeonRT: 3.49%

FPS: 0.00| 0.00 0.00| 0.00 0.00| 0.00 [18443010F1C4A90F00] [1.5] [3.651] [system] [info] Memory Usage - DDR: 109.69 / 337.18 MiB, CMX: 2.10 / 2.50 MiB, LeonOS Heap: 48.33 / 80.03 MiB, LeonRT Heap: 4.19 / 41.06 MiB

[18443010F1C4A90F00] [1.5] [3.651] [system] [info] Temperatures - Average: 31.10C, CSS: 32.25C, MSS 31.04C, UPA: 29.83C, DSS: 31.29C

[18443010F1C4A90F00] [1.5] [3.651] [system] [info] Cpu Usage - LeonOS 16.13%, LeonRT: 3.73%

FPS: 0.00| 0.00 0.00| 0.00 0.00| 0.00 [18443010F1C4A90F00] [1.5] [4.652] [system] [info] Memory Usage - DDR: 109.69 / 337.18 MiB, CMX: 2.10 / 2.50 MiB, LeonOS Heap: 48.33 / 80.03 MiB, LeonRT Heap: 4.19 / 41.06 MiB

[18443010F1C4A90F00] [1.5] [4.652] [system] [info] Temperatures - Average: 31.53C, CSS: 32.73C, MSS 30.80C, UPA: 30.80C, DSS: 31.77C

[18443010F1C4A90F00] [1.5] [4.652] [system] [info] Cpu Usage - LeonOS 15.87%, LeonRT: 4.02%

FPS: 0.00| 0.00 0.00| 0.00 0.00| 0.00 [18443010F1C4A90F00] [1.5] [5.653] [system] [info] Memory Usage - DDR: 109.69 / 337.18 MiB, CMX: 2.10 / 2.50 MiB, LeonOS Heap: 48.33 / 80.03 MiB, LeonRT Heap: 4.19 / 41.06 MiB

[18443010F1C4A90F00] [1.5] [5.653] [system] [info] Temperatures - Average: 31.65C, CSS: 32.97C, MSS 31.04C, UPA: 30.80C, DSS: 31.77C

[18443010F1C4A90F00] [1.5] [5.653] [system] [info] Cpu Usage - LeonOS 15.76%, LeonRT: 6.67%

FPS: 0.00| 0.00 0.00| 0.00 0.00| 0.00 [18443010F1C4A90F00] [1.5] [6.654] [system] [info] Memory Usage - DDR: 109.69 / 337.18 MiB, CMX: 2.10 / 2.50 MiB, LeonOS Heap: 48.33 / 80.03 MiB, LeonRT Heap: 4.19 / 41.06 MiB

[18443010F1C4A90F00] [1.5] [6.654] [system] [info] Temperatures - Average: 31.89C, CSS: 32.97C, MSS 31.53C, UPA: 30.80C, DSS: 32.25C

[18443010F1C4A90F00] [1.5] [6.654] [system] [info] Cpu Usage - LeonOS 15.51%, LeonRT: 4.55%

FPS: 0.00| 0.00 0.00| 0.00 0.00| 0.00 [18443010F1C4A90F00] [1.5] [7.655] [system] [info] Memory Usage - DDR: 109.69 / 337.18 MiB, CMX: 2.10 / 2.50 MiB, LeonOS Heap: 48.33 / 80.03 MiB, LeonRT Heap: 4.19 / 41.06 MiB

[18443010F1C4A90F00] [1.5] [7.655] [system] [info] Temperatures - Average: 31.71C, CSS: 33.21C, MSS 31.29C, UPA: 30.80C, DSS: 31.53C

[18443010F1C4A90F00] [1.5] [7.656] [system] [info] Cpu Usage - LeonOS 15.96%, LeonRT: 4.83%

FPS: 0.00| 0.00 0.00| 0.00 0.00| 0.00 [18443010F1C4A90F00] [1.5] [8.683] [system] [info] Memory Usage - DDR: 109.69 / 337.18 MiB, CMX: 2.10 / 2.50 MiB, LeonOS Heap: 48.33 / 80.03 MiB, LeonRT Heap: 4.19 / 41.06 MiB

[18443010F1C4A90F00] [1.5] [8.683] [system] [info] Temperatures - Average: 32.01C, CSS: 32.49C, MSS 31.53C, UPA: 32.01C, DSS: 32.01C

[18443010F1C4A90F00] [1.5] [8.683] [system] [info] Cpu Usage - LeonOS 15.86%, LeonRT: 5.10%

FPS: 0.00| 0.00 0.00| 0.00 0.00| 0.00 [18443010F1C4A90F00] [1.5] [9.684] [system] [info] Memory Usage - DDR: 109.69 / 337.18 MiB, CMX: 2.10 / 2.50 MiB, LeonOS Heap: 48.33 / 80.03 MiB, LeonRT Heap: 4.19 / 41.06 MiB

[18443010F1C4A90F00] [1.5] [9.684] [system] [info] Temperatures - Average: 32.31C, CSS: 33.69C, MSS 31.53C, UPA: 31.77C, DSS: 32.25C

[18443010F1C4A90F00] [1.5] [9.684] [system] [info] Cpu Usage - LeonOS 16.13%, LeonRT: 7.79%

FPS: 0.00| 0.00 0.00| 0.00 0.00| 0.00