M

mikegardner

Thanks Jaka, that's the conclusion I came to. The OAK 1 POE is suited well to direct POE Ethernet connections with a high bandwidth switch to a server or client. It doesn't perform well in a more conventional network topology, like an IP Camera does.
I think the better solution for my application is the OAK-D CM4, I can route the messages back via 4g with no bandwidth implications, and store the stream locally. When a message indicates an object of interest has been detected, I can stream back that section in an mp4 clip or similar.
Thanks for your input and help to understand the characteristics of the OAK1 POE product.
Mike

Hi Jaka,
Yes, I get a reasonable performance if I connect with a laptop directly to the USB C connector, or the directly to the ethernet connector. However in many applications, this is not a realistic use case, because there will be switches and routers between the client and the OAK 1 POE.
For example, if I get rid of the switch and replace it with a Ubiquiti 802.3AF PoE Injector that I purchased from your web store, connected directly into a Wifi router and then connect via Wifi to the laptop, using the Depthai Viewer I only get around 3 frames per second at 1080P, albeit with no dropout.
In this configuration, I get the following results from the bandwidth_test...
Downlink 48.3 mbps
Uplink 47.9 mbps

So in summary, (all at 1080P):

• Laptop connection via Wifi then the Netgear Prosafe FS 108P switch (10/100Mbps) = 1 frame displayed then drop-out

• Laptop connection to Wifi, then Ubiquiti POE injector from Wifi router to camera = 3 fps appx, no dropout

• Laptop connection via ethernet direct to the Netgear switch with the OAK also on the switch = 3 fps appx with occasional drop-out

• Laptop connection directly via Ethernet port to Ethernet port powered via USB C = 10 fps appx with occasional drop-out

• Laptop with direct USB C to USB C connection, seems like 30fps, no drop-out

My conclusion is that the performance is directly related to the connectivity, no surprises there I suppose.
However for many use cases there will rarely be direct Eth to Eth or USB to USB connectivity, and for my applicationI was hoping that I could use a 4g router connected to the camera and then a broadband connection from the laptop.

Given the raw network ping statistics are quite reasonable, I suspect the depthai_viewer is not well suited to these particular configuration above, and for a 4G use case I suppose I need to develop an app that compresses any video prior to sending it in an effort to improve the frame rate. In that case a OAK-D CM4 is probably a better candidate.
Mike

OK
with a direct PC to camera ethernet connection, I get
Downlink 891.7 mbps
Uplink 228.9 mbps
Press any key to continue...

However, with a connection via the switch, I get
Downlink 356.1 mbps
Traceback (most recent call last):
File "/home/mike/Luxonis/depthai-python/utilities/oak_bandwidth_test.py", line 91, in <module>
qin.send(buffer)
RuntimeError: Communication exception - possible device error/misconfiguration. Original message 'Couldn't write data to stream: 'xin' (X_LINK_ERROR)'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/usr/lib/python3.10/runpy.py", line 196, in _run_module_as_main
return _run_code(code, main_globals, None,
File "/usr/lib/python3.10/runpy.py", line 86, in _run_code
exec(code, run_globals)
File "/home/mike/Luxonis/depthai-python/utilities/oak_bandwidth_test.py", line 66, in <module>
with dai.Device(pipeline, maxUsbSpeed=dai.UsbSpeed.SUPER_PLUS) as device:
RuntimeError: Device already closed or disconnected: Input/output error
Stack trace (most recent call last):
#20 Object "[0xffffffffffffffff]", at 0xffffffffffffffff, in
#19 Object "python3", at 0x64627243d604, in _start
#18 Object "/lib/x86_64-linux-gnu/libc.so.6", at 0x7c47d1429e3f, in __libc_start_main
#17 Object "/lib/x86_64-linux-gnu/libc.so.6", at 0x7c47d1429d8f, in
#16 Object "python3", at 0x64627243d70c, in Py_BytesMain
#15 Object "python3", at 0x646272467182, in Py_RunMain
#14 Object "python3", at 0x646272475cd7, in Py_FinalizeEx
#13 Object "python3", at 0x6462724791d5, in
#12 Object "python3", at 0x64627233d05e, in
#11 Object "python3", at 0x6462724796c3, in
#10 Object "python3", at 0x64627234b63b, in
#9 Object "python3", at 0x64627234b840, in
#8 Object "/home/mike/.local/lib/python3.10/site-packages/depthai.cpython-310-x86_64-linux-gnu.so", at 0x7c47ceea44e3, in
#7 Object "/home/mike/.local/lib/python3.10/site-packages/depthai.cpython-310-x86_64-linux-gnu.so", at 0x7c47ceecc356, in
#6 Object "/home/mike/.local/lib/python3.10/site-packages/depthai.cpython-310-x86_64-linux-gnu.so", at 0x7c47cf12f178, in dai:evice::_Device()
#5 Object "/home/mike/.local/lib/python3.10/site-packages/depthai.cpython-310-x86_64-linux-gnu.so", at 0x7c47cf12ed63, in dai:evice::_Device()
#4 Object "/home/mike/.local/lib/python3.10/site-packages/depthai.cpython-310-x86_64-linux-gnu.so", at 0x7c47cf13a7e4, in dai:eviceBase::close()
#3 Object "/home/mike/.local/lib/python3.10/site-packages/depthai.cpython-310-x86_64-linux-gnu.so", at 0x7c47cf132f04, in dai:evice::closeImpl()
#2 Object "/home/mike/.local/lib/python3.10/site-packages/depthai.cpython-310-x86_64-linux-gnu.so", at 0x7c47cf14f904, in dai:eviceBase::closeImpl()
#1 Object "/home/mike/.local/lib/python3.10/site-packages/depthai.cpython-310-x86_64-linux-gnu.so", at 0x7c47cf13ffaa, in dai:eviceBase::hasCrashDump()
#0 Object "/home/mike/.local/lib/python3.10/site-packages/depthai.cpython-310-x86_64-linux-gnu.so", at 0x7c47cf17331d, in nanorpc::core::client<nanorpc::packer::nlohmann_msgpack>::result nanorpc::core::client<nanorpc::packer::nlohmann_msgpack>::call<>(unsigned long)
Segmentation fault (Address not mapped to object [0x10])
Segmentation fault (core dumped)

Hi Jaka
I've just seen your reply to syg regarding bandwidth.

Looking at the link https://docs.luxonis.com/software/depthai/optimizing/#Bandwidth

What Resolution FPS FPS set PoE Latency [ms] USB Latency [ms] Bandwidth
Color (isp) 1080P 25 25 51 33 Std: 0.8 622 Mbps

unless I'm misunderstanding the guidelines, my set-up at 30fps colour 1080P is likely to require in excess of 622 Mbps! That's almost certainly the problem, I'm using a Netgear Prosafe FS108P which is 10/100Mbps each port.
Thanks

Here are the logs when the device is connected via the POE switch.

MESA-INTEL: warning: Haswell Vulkan support is incomplete
Backend started successfully.
[2024-09-23T10:38:49Z INFO ewebsock::native_tungstenite] WebSocket handshake has been successfully completed
[2024-09-23T10:38:49Z INFO re_viewer::depthai::ws] Websocket opened
Selecting device: 18443010B1E752F500
Connecting to viewer at 127.0.0.1:9876
Oak cam: <depthai_sdk.oak_camera.OakCamera object at 0x7e5dbf06d150>
Default stereo pair: None
Pipeline config: auto=True cameras=[CameraConfiguration(fps=30, resolution=<CameraSensorResolution.THE_1080_P: 'THE_1080_P'>, kind=<CameraSensorType.COLOR: 0>, board_socket=<CameraBoardSocket.CAM_A: 0>, stream_enabled=True, name='Color', tof_align=None)] stereo=None ai_model=AiModelConfiguration(display_name='Yolo V6', path='yolov6nr3_coco_640x352', camera=<CameraBoardSocket.CAM_A: 0>) imu=ImuConfiguration(report_rate=100, batch_report_threshold=5)
Updating pipeline...
Creating auto pipeline config
Connected camera features: [{socket: CAM_A, sensorName: IMX378, width: 4056, height: 3040, orientation: AUTO, supportedTypes: [COLOR], hasAutofocus: 1, hasAutofocusIC: 1, name: color}]
Usb speed: UsbSpeed.UNKNOWN
Connected to a PoE device, camera streams will be JPEG encoded...
Creating camera: fps=30 resolution=<CameraSensorResolution.THE_1080_P: 'THE_1080_P'> kind=<CameraSensorType.COLOR: 0> board_socket=<CameraBoardSocket.CAM_A: 0> stream_enabled=True name='color' tof_align=None
Connected cam doesn't have IMU, skipping IMU creation...
Starting pipeline
Got message to send: <depthai_viewer.backend.messages.WarningMessage object at 0x7e5dbfaf7be0>

Sending message: <depthai_viewer._backend.messages.WarningMessage object at 0x7e5dbfaf7be0>

[0s] System information

Ddr used / total - 96.90 / 333.28 MiB
Cmx used / total - 2.49 / 2.50 MiB
LeonCss heap used / total - 41.00 / 82.20 MiB
LeonMss heap used / total - 4.36 / 40.35 MiB
Chip temperature - average: 31.64, css: 33.69, mss: 30.08, upa: 31.29, dss: 31.53
Cpu usage - Leon CSS: 16.14 %, Leon MSS: 0.04 %

[2024-09-23T10:39:55Z WARN re_viewer::depthai::depthai] Device is connected via PoE. This may cause performance issues.

[19s] System information

Ddr used / total - 109.08 / 333.28 MiB
Cmx used / total - 2.49 / 2.50 MiB
LeonCss heap used / total - 41.89 / 82.20 MiB
LeonMss heap used / total - 5.21 / 40.35 MiB
Chip temperature - average: 32.73, css: 34.40, mss: 31.77, upa: 32.97, dss: 31.77

Cpu usage - Leon CSS: 26.49 %, Leon MSS: 0.42 %

[27s] System information

Ddr used / total - 109.08 / 333.28 MiB
Cmx used / total - 2.49 / 2.50 MiB
LeonCss heap used / total - 41.89 / 82.20 MiB
LeonMss heap used / total - 5.21 / 40.35 MiB
Chip temperature - average: 33.50, css: 34.88, mss: 32.97, upa: 34.16, dss: 32.01
Cpu usage - Leon CSS: 27.33 %, Leon MSS: 0.42 %
^C[2024-09-23T10:40:41Z WARN re_sdk_comms::server] Closing connection to client: early eof

Thanks Jaka

depthai_viewer Version is

0.2.4 [rustc 1.74.1 (a28077b28 2023-12-04), LLVM 17.0.4] x86_64-unknown-linux-gnu bf0ca8b, built 2024-06-21T10:37:38Z

With the USB C connection I get the following report when launching
MESA-INTEL: warning: Haswell Vulkan support is incomplete
Backend started successfully.
[2024-09-23T10:24:32Z INFO ewebsock::native_tungstenite] WebSocket handshake has been successfully completed
[2024-09-23T10:24:32Z INFO re_viewer::depthai::ws] Websocket opened
Selecting device: 18443010B1E752F500
Connecting to viewer at 127.0.0.1:9876
Oak cam: <depthai_sdk.oak_camera.OakCamera object at 0x792dc8101120>
Default stereo pair: None
Pipeline config: auto=True cameras=[CameraConfiguration(fps=30, resolution=<CameraSensorResolution.THE_1080_P: 'THE_1080_P'>, kind=<CameraSensorType.COLOR: 0>, board_socket=<CameraBoardSocket.CAM_A: 0>, stream_enabled=True, name='Color', tof_align=None)] stereo=None ai_model=AiModelConfiguration(display_name='Yolo V6', path='yolov6nr3_coco_640x352', camera=<CameraBoardSocket.CAM_A: 0>) imu=ImuConfiguration(report_rate=100, batch_report_threshold=5)
Updating pipeline...
Creating auto pipeline config
Connected camera features: [{socket: CAM_A, sensorName: IMX378, width: 4056, height: 3040, orientation: AUTO, supportedTypes: [COLOR], hasAutofocus: 1, hasAutofocusIC: 1, name: color}]
Usb speed: UsbSpeed.SUPER
Creating camera: fps=30 resolution=<CameraSensorResolution.THE_1080_P: 'THE_1080_P'> kind=<CameraSensorType.COLOR: 0> board_socket=<CameraBoardSocket.CAM_A: 0> stream_enabled=True name='color' tof_align=None
Connected cam doesn't have IMU, skipping IMU creation...

Starting pipeline

[0s] System information

Ddr used / total - 96.89 / 333.28 MiB
Cmx used / total - 2.49 / 2.50 MiB
LeonCss heap used / total - 23.21 / 82.20 MiB
LeonMss heap used / total - 4.26 / 40.35 MiB
Chip temperature - average: 30.86, css: 33.21, mss: 29.59, upa: 30.08, dss: 30.56

Cpu usage - Leon CSS: 4.49 %, Leon MSS: 0.05 %

[10s] System information

Ddr used / total - 96.89 / 333.28 MiB
Cmx used / total - 2.49 / 2.50 MiB
LeonCss heap used / total - 24.10 / 82.20 MiB
LeonMss heap used / total - 4.92 / 40.35 MiB
Chip temperature - average: 35.17, css: 37.48, mss: 34.16, upa: 35.11, dss: 33.93
Cpu usage - Leon CSS: 42.71 %, Leon MSS: 26.02 %

I'll post the logs with the POE connection shortly.

OK - I've deleted all of the folders and recloned them and re-installed depthai

This has enabled me to use the device manager to update the firmware to 0.0.28 but it still behaves the same way. I've tried the depthai_viewer using the USB C port connected to my laptop and everything works fine.

I'll try again with a different POE switch or injector.

Thanks Jaka. I've deleted all of the folders and re-cloned them at the latest level and re-installed depthai.
Please consider my query on this thread closed now.
Regarding the drop-out I'm experiencing- I'll continue on my original thread.

mikegardner
Here's a sample of the ping responses, IMHO they don't suggest a network issue...

PING 192.168.1.133 (192.168.1.133) 56(84) bytes of data.
64 bytes from 192.168.1.133: icmp_seq=1 ttl=64 time=241 ms
64 bytes from 192.168.1.133: icmp_seq=2 ttl=64 time=4.16 ms
64 bytes from 192.168.1.133: icmp_seq=3 ttl=64 time=3.64 ms
64 bytes from 192.168.1.133: icmp_seq=4 ttl=64 time=8.02 ms
64 bytes from 192.168.1.133: icmp_seq=5 ttl=64 time=3.86 ms
64 bytes from 192.168.1.133: icmp_seq=6 ttl=64 time=4.24 ms
64 bytes from 192.168.1.133: icmp_seq=7 ttl=64 time=3.78 ms
64 bytes from 192.168.1.133: icmp_seq=8 ttl=64 time=3.30 ms
64 bytes from 192.168.1.133: icmp_seq=9 ttl=64 time=14.9 ms
64 bytes from 192.168.1.133: icmp_seq=10 ttl=64 time=3.86 ms
64 bytes from 192.168.1.133: icmp_seq=11 ttl=64 time=4.27 ms
64 bytes from 192.168.1.133: icmp_seq=12 ttl=64 time=3.36 ms
64 bytes from 192.168.1.133: icmp_seq=13 ttl=64 time=3.16 ms
64 bytes from 192.168.1.133: icmp_seq=14 ttl=64 time=3.92 ms
64 bytes from 192.168.1.133: icmp_seq=15 ttl=64 time=2.95 ms
64 bytes from 192.168.1.133: icmp_seq=16 ttl=64 time=3.47 ms
64 bytes from 192.168.1.133: icmp_seq=17 ttl=64 time=2.60 ms
64 bytes from 192.168.1.133: icmp_seq=18 ttl=64 time=7.16 ms
64 bytes from 192.168.1.133: icmp_seq=19 ttl=64 time=2.65 ms
64 bytes from 192.168.1.133: icmp_seq=20 ttl=64 time=2.73 ms
64 bytes from 192.168.1.133: icmp_seq=21 ttl=64 time=4.29 ms
64 bytes from 192.168.1.133: icmp_seq=22 ttl=64 time=3.84 ms
64 bytes from 192.168.1.133: icmp_seq=23 ttl=64 time=2.77 ms
64 bytes from 192.168.1.133: icmp_seq=24 ttl=64 time=3.12 ms
64 bytes from 192.168.1.133: icmp_seq=25 ttl=64 time=2.62 ms
64 bytes from 192.168.1.133: icmp_seq=26 ttl=64 time=2.43 ms
64 bytes from 192.168.1.133: icmp_seq=27 ttl=64 time=4.75 ms

jakaskerl

Thanks for your assistance

mikegardner
Can you give more info on OS you are using?

I'm using Ubuntu 22.04.5 LTS

mikegardner I used the device manager using the link on the docs page to try to update the firmware using the Danger Zone 'Update Bootloader' and 'Flash Factory Bootloader' options but the 'Flashed bootloader version' always stays at 0.0.21

This would suggest you are using an older version of depthai or that the flashing failed (in which case you would get an error).

I suspected I was using an older version of depthai, so I deleted the depthai, depthai-python, and depthai-core folders and cloned them again from the master branch on the current repo, ran submodule update --init --recursive on each of those folders plus run install_requirements git but still got the same results.

mikegardner I'm up to date with the depthai repo but no idea where the bootloader file is, it's not in the utilities directory with the device_manager.py module

But it should be. luxonis/depthai-pythontree/main/utilities

Looking at the repo at the link you suggested, it matches my local folder contents exactly. As i say, I can see the device_manager.py module but nothing that looks like the actual bootloader file that is installed.

mikegardner If I ping the camera I get on average 10-15ms response with no missed beats

But this would indicate a network issue. What does your setup look like?

I have a Netgear Prosafe 108P 8-port switch with 4 POE ports. The only POE port that is populated is the one for the camera. The switch is connected to my Wifi router and I use a a Thinkpad T440s running Unbuntu 22.04.5 LTS to attach to the camera.

The laptop is dedicated as a Depthai dev system

Thanks,
Jaka

syg

Hi Syg. Could you share how you updated the firmware. I followed the instructions in the link provided by jakaskeri

(https://docs.luxonis.com/software/depthai-components/bootloader/#Danger%20Zone)

but I can't get past 0.0.26 or depthai version 2.24.0.0

I've deleted the depthai and depthai-python foldesr then cloned it from the current repo master, I've followed the docs again, as though starting with a clean install, installed depthai, installed all requirements. The utility flashed the camera but still at 0.0.26

I'm at a loss to understand why my depthai version won't get past 2.24.0.0 when using the latest repo clone.

I've just purchased an OAK-1 POE device and followed the setup as advised in the docs, then run the DepthAI Viewer.

On launch, the terminal window reports "Device is connected via POE, This may cause performance issues."

The GUI takes about a minute to find the device, shows a single frame, then drops out. At that point, the terminal window reports "ping was missed, closing the device connection" followed by "Flashed bootloader version 0.0.21, less than 0.0.28 is susceptible to boot/restart failure. Upgrading is advised, flashing main/factory (not user) bootloader. Available 0.0.28"

I used the device manager using the link on the docs page to try to update the firmware using the Danger Zone 'Update Bootloader' and 'Flash Factory Bootloader' options but the 'Flashed bootloader version' always stays at 0.0.21

I'm up to date with the depthai repo but no idea where the bootloader file is, it's not in the utilities directory with the device_manager.py module

If I ping the camera I get on average 10-15ms response with no missed beats

Any advice appreciated

OK Jaka

With respect to the tools - Is there any preference on the model type, eg MobileNet-SSD or Yolo, and if Yolo, which version performs best with the tools.

Thanks

Thanks for the explanation Jaka, I've just ordered a OAK1 POE which I believe is RVC4 based. All of the conversions I tried previously were in 2021, so hopefully the tools have matured since then.

I'll create my model and follow the documentation to convert it to a blob and see how it goes.

Thanks

Hi Everyone, I'm enquiring about the robustness of the latest blob converter tools

I dabbled with all of the OAK (OAK 1, OAK D and the Pi ) hardware a few years ago and found it excellent quality and a great out of the box experience with the pre-trained blobs that were available from Luxonis.

However, at that time the blob conversion was quite flaky, and I lost interest in using the platform for deploying custom image sets.

I now have a commercial project that is extremely well suited to the OAK 1 or a similar custom single camera and I'd like to explore using the conversion tools again.

Can anyone testify to their experience of using performing a complete end-to-end capture, training, validation and testing and most importantly conversion to blob and deployment to the camera?

Thanks in advance

themarpe Thanks a lot for the comprehensive explanation Martin. I'll try those suggestions out.

I tried the same file with VS Code straight out of the box and it's different. Here are a few of the entries in the VS Code problem log.

"message": "\"node\" is not a known member of module",
"message": "Cannot access member \"setStreamName\" for type \"AprilTag\"
"message": "Cannot access member \"setBoardSocket\" for type \"AprilTag\"
"message": "Cannot access member \"setResolution\" for type \"AprilTag\"
"message": "Cannot access member \"setMaxOutputFrameSize\" for type \"AprilTag\"
"message": "Cannot access member \"getVideoWidth\" for type \"AprilTag\"
"message": "Cannot access member \"getVideoHeight\" for type \"AprilTag\"

There are a quite a few more entries in practice which suggests to me that VS Code requires some manual configuration to get the indexer set up properly. So the VS Code aspect may be a distraction, as it works in my depthai environment for C++ .

I would like to get PyCharm setup correctly as it is recommended as the IDE of choice in the installation section...

https://docs.luxonis.com/projects/api/en/latest/install/?highlight=pycharm#test-installation

For development machines like Mac/Windows/Ubuntu/etc., we recommend the PyCharm IDE, as it automatically makes/manages virtual environments for you, along with a bunch of other benefits.

As I say it's a minor annoyance rather than a real issue, as all of the code works. It's just bothers me that something's not quite right with the indexing, and I'd like to understand it in order to fix it.

Any thoughts appreciated.

Hi Erik, I think I'm using the very latest version, I re-installed a couple of days ago to see if it made a difference, using the instructions from the site. https://docs.luxonis.com/projects/api/en/latest/install/
The version showing in the PyCharm 'Python Packages' list box is depthai 2.15.2.0

I'll try the above edge_detector.py example on VS Code and see how that behaves, and report back. Thanks.

ZigaKerzan Thanks for the prompt response.