Connection gets broken randomly

GeonsooKim

OS: Ubuntu 20.04
Camera: OAK-D-PoE Pro
Depthai version: 2.17.4.0
FW version: 0.0.20

I am experiencing an issue that the camera sometimes becomes unrecognizable after I run my script N times. More specifically, I get 0 for depthai.Device.getAllAvailableDevices() which means no OAK camera is found even though it's physically connected and has been used for N times. Whenever this happens, I just disconnect the cable and connect back in. Then, the issue gets resolved. Both camera and host PC are connected to an Ethernet switch and they are only devices connected to the switch.

This happened twice for last 5 hours so far, so I don't see this issue often. As it happens randomly, it's not reproducible on purpose, which makes it hard to debug. Do you have any idea on this?

One thing is that I use control+backslash (Core Dump) in terminal to terminate my program. Does it make an unclean exit and make an effect on FW routine?

I much appreciate your feedback. Thank you.

erik

Hi GeonsooKim ,
Yes exiting that way in terminal surely doesn't exit the device correctly (perform the device.close()), but I thin the watchdog on OAK POE camera should kick in and reset it into a correct state after some time (10 seconds?). Do you wait some time before trying to connect again? Could you also try out a suggestion from “Special” network equipment from our poe debugging docs?
Thanks, Erik

GeonsooKim

Erik,

Yesterday, when I had the issues, I expected the watchdog kicks in too, but it did not happen even though I waited for 10₂₀ mins each time. So, I had to disconnect the cable and connect it back to make the camera recognizable again.

Thanks for the advice. I will try increasing the timeout/ adding delay next time it happens and see how it behaves.

Thanks, Geonsoo

erik

GeonsooKim I have just tried on ubuntu 20.04, with OAK-D-PoE, depthai 2.17.4, bootloader 0.20 flashed. Did the core dump + restart after 15 seconds 10 times in a row and I can't reproduce this issue. I have the OAK POE connected directly to my laptop with ethernet cable - no router in between (and powering via USB). How does your network setup look like?

GeonsooKim

erik Thanks for taking time to try it. Here's my network setup.

OAK-D PoE Pro is connected to an Ethernet switch
Host PC (my laptop) is connected to the Ethernet switch
the Ethernet switch is connected to main network hub

The issue that camera becomes unrecognizable happened randomly and rarely (2 times for 6 hours). I wasn't able to reproduce it on purpose either.

FYI. This is the script I used.

import cv2
import depthai as dai
from test_utils import *

# Create pipeline
pipeline = dai.Pipeline()

# Define source and output
camRgb = pipeline.create(dai.node.ColorCamera)
camRgb.setColorOrder(dai.ColorCameraProperties.ColorOrder.RGB)
camRgb.setResolution(dai.ColorCameraProperties.SensorResolution.THE_4_K)  # THE_1080_P, THE_4_K, THE_12_MP

xoutRgb = pipeline.create(dai.node.XLinkOut)
xoutRgb.setStreamName("rgb")

# Linking
camRgb.isp.link(xoutRgb.input)

print(camRgb.getResolution())
w, h = camRgb.getResolutionSize()
print(w, h)
# print(camRgb.getIspSize())
print("booting")
# Connect to device and start pipeline
with dai.Device(pipeline) as device:
    print("up")

    # Output queue will be used to get the rgb frames from the output defined above
    qRgb = device.getOutputQueue(name="rgb", maxSize=1, blocking=False)  # depthai.DataOutputQueue

    while True:
        
        # wait_beam_trigger()
        # imRgb = qRgb.get()  # blocking call, will wait until a new data has arrived
        imRgb = qRgb.tryGet()  # non-blocking call  # depthai.ImgFrame
        if imRgb is not None:
            im = imRgb.getCvFrame()  # numpy.ndarray
            down_size = (w//8, h//8)
            im_down = cv2.resize(im, down_size, interpolation=cv2.INTER_LINEAR)
            print(im.shape, im_down.shape)
            cv2.imshow("rgb", im)
            cv2.imshow("rgb_down",im_down)
            cv2.waitKey()
            cv2.destroyAllWindows()  # cv2.destroyWindow("rgb")

RobertCarroll

Hey Guys,

Did you find a solution I have the same issue and need any advice .

Thanks in advance

Rob C

jakaskerl

Hi @RobertCarroll
Sorry, I'm out of office atm and don't have access to POE devices. I'll test it when I get back.

RobertCarroll

Hey Jakaskerl,

After some investigation on two separate poe switches seems to be related to the cables. When I move the cables the poe disconnects and both cameras go down.

The cables are the ones that came with the kit is there any known issues with these cables ? I am using TSW100 - Industrial PoE+ Switch without Management

The TSW100 is the first industrial switch from Teltonika Networks without management.
Ideal for professional high bandwidth applications to provide both reliable data connections and the required power supply.

5x RJ45 Gigabit Ethernet 10/100/1000 (4xPoE+)
120 W PoE total power budget
Mounting on DIN rail or wall

PoE
Four PoE ports with 802.3af and 802.3at support

Durability
Rugged aluminum housing

Mounting
DIN rail and surface mount options

Power consumption
Total power budget at PSE up to 120 W

Ethernet
Five Gigabit Ethernet interfaces with speeds of up to 1,000 Mbit/s

Plug & Play
No additional configuration required

I am not sure to why the camera reset but it seems to be a quality issue on the cables can anyone confirm this and I will need to source some new ones ?

jakaskerl

Hi @RobertCarroll
There should not be any issues with the cables (at least none that I experienced). But it might be worth checking with a different cable if you can. And make sure the connectors are secured and have proper contact.

Thanks,
Jaka

RobertCarroll

Hey Jakaskerl,

The connections are secure and the issue happens on two separate switch's indicating the cables to be at fault. I have to order two new cables to be sure and plan to over the weekend.

I can investigate once they arrive and get back and close this tread.

Thanks again

Rob C