Hi! We are having issues when connecting multiple (4) cameras (PoE). Sometimes we are able to ping some of the cameras and often it varies which we are able to ping. Each camera has been configured to use a static IP address. We have verified the PoE switch is able to deliver enough power, we have flashed the cameras to use the latest firmware, we have also tried multiple network cards. Any ideas of what could cause the instability? Camera: Oak 1 PoE OS: Windows 11 Thanks!

Hi @"Kristoffer"#1750 , We have seen that usually the culprit of such problem is the mask/gateway. Were those 2 fields configured correctly when flashing the static IP?

Mask should be OK, but is the gateway also required?

Hi @"Kristoffer"#p14812 Could you post a screenshot of the device manager? Just so we can verify. Thanks, Jaka

[URL=https://i.imgur.com/yHGvqy1.png][IMG]https://i.imgur.com/yHGvqy1.png[/IMG][/URL] [URL=https://i.imgur.com/c57uavy.png][IMG]https://i.imgur.com/c57uavy.png[/IMG][/URL] [URL=https://i.imgur.com/6ofv5yC.png][IMG]https://i.imgur.com/6ofv5yC.png[/IMG][/URL]

Hi @"Kristoffer"#1750 , Mask should likely be 255.255.0.0, and if you want to access internet from the device you should set correct gateway as well.

Hi, 255.255.255.0 (most common) should be fine for our configuration and we do not need access to the internet from the cameras. Anything else that could cause the problem?

We are still having problems connecting to the cameras. We are running 4 x OAK1-PoE, Windows 11 and have configured each camera to use static IP. Often we are unable to ping and which camera varies, sometimes we are able to ping, but not start the camera. Often we need to retry multiple times and eventually all 4 cameras will run. After struggling for a long time (not getting the cameras to run) a reboot of the host PC does something, but we still need to try for a long time to get the cameras running. Are there some specific setting in Windows we need to change? Do you have examples of how you run multiple cameras and which hardware? Do we need to fill in the MAC address field and what value should we use and for what purpose? Thanks!

Hi Kristoffer! There could be several reasons why you're having trouble connecting to your cameras. Here are some suggestions: 1. **Check the Network**: Make sure that all the cameras are powered on and connected to the same network as your computer. If you are using a DHCP server, you can check the connected devices on the DHCP server dashboard to see if the camera is connected and its IP. If there is no DHCP, the camera will fallback to a static IP. 2. **Increase the Timeout**: With certain network equipment, the default 15 seconds timeout for connecting to the POE device might be insufficient and needs to be increased. You can increase the timeout by using environmental variables. For example, in Windows CMD, you can use the following commands: ```cmd set DEPTHAI_WATCHDOG_INITIAL_DELAY=60000 set DEPTHAI_BOOTUP_TIMEOUT=60000 python3 script.py ``` 3. **Check the Bootloader Version**: Make sure that the camera has bootloader version 0.0.18 or newer flashed. You can use the OAK Device Manager to check this. 4. **Check the Network Interface Controller Settings**: Some default NIC settings on Linux might not be ideal for communication with OAK POE cameras, which can result in slow FPS, high latency, and/or high OAK CPU usage. You can use `ethtool` to configure these settings. 5. **Disconnect from One of the Interfaces**: If your host computer is connected to the same LAN, it can happen that device discovery finds the same PoE device twice, leading to an error. Disconnecting from one of the interfaces, such as WiFi, should resolve this issue. 6. **Check the Firewall**: UDP Device discovery is handled on port `11491`, and TCP XLink connection is handled on port `11490`. If you have your firewall enabled, you might need to allow these two ports. 7. **Specify the MAC Address**: You can flash static/dynamic IP of an OAK-PoE device and also specify DNS and MAC address. However, the MAC address is not included in the demo. As for running multiple cameras, you can use the DepthAI library to search for available OAK PoE cameras inside the same LAN. You can specify the POE device to be used by the IP address as well. Regarding the choice of operating system, Linux might provide better control over network settings compared to Windows. However, both should work fine for running the cameras. Sources: - [Getting Started with PoE](https://docs.luxonis.com/projects/hardware/en/latest/pages/guides/getting-started-with-poe#no-depthai-devices-found) - [Network Interface Controller Settings](https://docs.luxonis.com/projects/hardware/en/latest/pages/guides/getting-started-with-poe#network-interface-controller-settings) - [Flash Static IP](https://docs.luxonis.com/projects/hardware/en/latest/pages/guides/getting-started-with-poe#flash-static-ip) - [Special Network Equipment](https://docs.luxonis.com/projects/hardware/en/latest/pages/guides/getting-started-with-poe#special-network-equipment) - [I Can Ping the OAK PoE Camera, But Can’t Connect to It](https://docs.luxonis.com/projects/hardware/en/latest/pages/guides/getting-started-with-poe#i-can-ping-the-oak-poe-camera-but-can-t-connect-to-it) - [Connected to the Same LAN via 2 Interfaces (WiFi, Ethernet)](https://docs.luxonis.com/projects/hardware/en/latest/pages/guid Hope this helps! :) Best, Luxonis BOT

Hi @"Kristoffer"#p14989 Could you try with this script: https://discuss.luxonis.com/d/2241-sometimes-cant-connect-when-using-multiple-cameras/3 Let me know if it works. Thanks, Jaka

Hi! Yes, but it is still not working reliably. Sometimes the cameras with their assigned IP becomes invisible. Eg. A camera may be configured to 192.168.10.60 and will show up when pinging and it is also possible to connect normally (at least for a few tries). Then it suddenly becomes invisible from ping and dai.Device.getAllAvailableDevices(). However, when changing the IP of the network interface to be 169.254.1.50, all cameras show up again (all with the default IP as name, even if all cameras are configured to a unique static IP). ### Network adapter configured to 192.168.10.94 [DeviceInfo(name=192.168.10.93, mxid=18443010B1410E0900, X_LINK_BOOTLOADER, X_LINK_TCP_IP, X_LINK_MYRIAD_X, X_LINK_SUCCESS), DeviceInfo(name=192.168.10.90, mxid=1844301081145C1200, X_LINK_BOOTLOADER, X_LINK_TCP_IP, X_LINK_MYRIAD_X, X_LINK_SUCCESS)] ### Network adapter configured to 169.254.1.50 [DeviceInfo(name=169.254.1.222, mxid=1844301081145C1200, X_LINK_BOOTLOADER, X_LINK_TCP_IP, X_LINK_MYRIAD_X, X_LINK_SUCCESS), DeviceInfo(name=169.254.1.222, mxid=18443010B1410E0900, X_LINK_BOOTLOADER, X_LINK_TCP_IP, X_LINK_MYRIAD_X, X_LINK_SUCCESS), DeviceInfo(name=169.254.1.222, mxid=14442C10610DCCD600, X_LINK_BOOTLOADER, X_LINK_TCP_IP, X_LINK_MYRIAD_X, X_LINK_SUCCESS), DeviceInfo(name=169.254.1.222, mxid=18443010B1975B1200, X_LINK_BOOTLOADER, X_LINK_TCP_IP, X_LINK_MYRIAD_X, X_LINK_SUCCESS)] I believe there is something wrong with the implementation of device_info = dai.DeviceInfo(IP) and dai.Device(device_info). Connecting using MXID works more reliably.

Unable to ping multiple cameras

jakaskerl

Hi Kristoffer
Are you running the latest bootloader 0.0.26? From the screenshot above it looks like you are using *24. Would you mind updating first, since many issues like that were fixed in the patch.

Thanks,
Jaka

Kristoffer

Hi!

I have now updated to the latest bootloader and also use a hybrid of your script to wait for all devices to be in state X_LINK_BOOTLOADER. I then try to connect using IP, but the problem remains. It may start up all four cameras without any problems the first time, but when the camera/script is shut down, it has a really hard time connecting to the cameras using the IP even if all the devices are found using dai.Device.getAllAvailableDevices() and waiting for all devices to be in the X_LINK_BOOTLOADER state. It has to retry multiple times before the cameras start running and eventually it will stop finding some cameras.

Kristoffer

We still have huge problems starting multiple devices (4) and it looks to me that it is something weird happening when trying to start the script over remote desktop. The devices do usually start much more easily when running the script locally, but we can not do that as some of our customers are located far away.

The devices are of the type OAK-1-PoE and we are running Windows 11.

We have tried the following:

Starting devices using threads
Starting devices sequentially
We have used both our own code and your example code for starting multiple devices
Using a separate network card for each device
Using a switch instead of separate network cards
Creating a routing table in Windows
Connecting using IP (static IP on device)
Connecting using MXID
Increasing the watchdog timeout to 60 seconds
Using the famous if device_info.state != dai.XLinkDeviceState.X_LINK_BOOTLOADER
Deactivating IPv6
Deactivating Wifi
Updating depthai
Updating the bootloader to the latest version (multiple times)

Can you do some testing and see if you get the same problems when connecting over remote desktop? The devices are configured with static IP's.

jakaskerl

Hi Kristoffer
Ok, thanks. Will try to debug this on Monday as I currently don't have access to a win11 machine.

jakaskerl

Hi Kristoffer
Do you have any MRE I could run locally that would reproduce the issue you are experiencing? I will try to run through remote desktop.

Thanks,
Jaka

Kristoffer

Hi!

The problems start already before the script is run it seems. And we are using Windows 10 apparently, not 11 as stated earlier.

The problem is mainly that Windows have problems finding the four devices between reboots. When running the device manager from the depthai utilities only 3 or maybe 2 devices will show. We are not even able to ping the missing devices.

Lets say we have device1, device2, device3 and device4. After booting only device1,device2 and device4 are showing (not device3). The next time we boot device1 could be the one missing, but device3 and the rest is found. Then suddenly after connecting to the devices only two will show.

jakaskerl

Hi Kristoffer
Can you confirm you are only getting this issue when running on windows?
UDP Device discovery (port 11491) is likely the cause here, any way you could check the network traffic?

Thanks,
Jaka

Kristoffer

The firewall is now turned completely off and we still experience the same problems. This is a problem happening when using Windows (10 in our case) and when running multiple devices (4 in our case).

Windows is not able to detect all cameras from the device_manager utility and this happens right after Windows has booted usually. Right now it is only detecting 2 devices, but sometimes it will detect all 4 or 3. Trying to connect to the devices usually makes it even worse when we shut them down and then try to reconnect after waiting for the devices for at least 10 seconds to boot up of course.

Are you able to test running at least 4 devices (OaK-1-PoE) running Windows 10? This has been going on for weeks now and nothing has changed. I installed Depthai 24 earlier this week without any change.

We really need this fixed and have even sent an email to support explaining how critical this is for us without even getting a reply.

Thanks

jakaskerl

Kristoffer Are you able to test running at least 4 devices (OaK-1-PoE) running Windows 10?

Yes, I am. On two separate machines.

I ran https://docs.luxonis.com/projects/api/en/latest/samples/mixed/multiple_devices/ 10 times with 10/10 success rate. I'm unsure what the issue is on your side. I have had problems with autodiscovery when devices shared the same IP, but you stated this is not the case for you. Do you have a DHCP server connected to the switch that would assign each device its own unique IP (I had them freshly unboxed, didn't update bootloader, didn't flash static IP, but I connected them via switch to DHCP)?

I did have issue when restarting the script directly after, but if I waited 10 seconds (giving devices time to properly reboot), the issue was gone.

Thanks,
Jaka

Kristoffer

HI!

Great! Then I at least know it should be possible which is a step forward. We have tried using both static IP and a configured DHCP server. Do you use the internal Windows DHCP server or a third party program?

May you also share which hardware components you are using on the machines and exactly which Windows version you are using? Maybe it is a hardware issue. I did try the prebuilt Luxonis Raspberry Pi image and were able to run the devices successfully multiple times in a row however.

I am also interested in how you installed Depthai and its dependencies. There is an MSI installer and Chocolatey is also mentioned as an alternative. We used Conda and downloaded Depthai using the following environment.yml file:

name: depthai_24
channels: # Channels are listed in prioritized order, do not change!
- conda-forge
- defaults
dependencies:
- python>=3.12,<3.13
- numpy
- opencv
# Pysimplegui and pillow required by DepthAI utilities (eg. device manager)
- pysimplegui
- pillow
- pip
- pip:
# Install prebuilt wheel (win_amd64) from:
- https://artifacts.luxonis.com/artifactory/luxonis-python-snapshot-local/depthai/depthai-2.24.0.0.dev0+3f39768ab74394fa629ad6b4bfbf4d9a9f99eb6c-cp312-cp312-win_amd64.whl
variables:
DEPTHAI_WATCHDOG: 4500
DEPTHAI_PROTOCOL: tcpip

Thanks!

jakaskerl

Hi Kristoffer

Kristoffer DEPTHAI_PROTOCOL: tcpip

Don't use environment variables. Especially not the protocol. The device discovery is UDP, which might be blocked after using this setting.

Kristoffer Do you use the internal Windows DHCP server or a third party program?

DHCP on a router connected to the switch.

Kristoffer I am also interested in how you installed Depthai and its dependencies.

Installed via installer (so dependencies are installed). Then went to some custom directory, pulled depthai-python, did install_requirements.py inside examples/ and then inside /utilities as well. Ran python examples/mixed/multiple_devices.py.

EDIT: also used python3.11 since 3.12 is not currently supported unless on the python312 branch.

Thanks,
Jaka

« Previous Page