RuntimeError: Communication exception

MhmdBarazi · Aug 28, 2023

Hello dear@"jakaskerl"#p11449,

I tried from a different device, with and without callback. Still the same problem

MhmdBarazi · Aug 28, 2023

Update, There is a non-continuous motion in the window(sometimes it freezes) as well as it is too slow.

jakaskerl · Aug 28, 2023

Hi MhmdBarazi
Upon testing the model locally, I believe it's to computationally expensive. Just as you have experienced above, I am also only getting about 0.5 FPS. This is not the case if I'm using other models.

Thoughts?
Jaka

MhmdBarazi · Aug 28, 2023

I am relatively new in this domain unfortunately. But I tested the previous model on the other PC, it gave me a similar performance. What can you suggest to me jakaskerl ?

Thanks

jakaskerl · Aug 28, 2023

Hi MhmdBarazi
The problem is OAK device processing speed, that's why the results are the same regardless of the host you are using. I believe you would have to scale down the model (set input to 300x300 instead of 640x640, which is very high) to achieve faster inference, perhaps change the framework (iirc yolo6 runs the fastest)..

Thanks,
Jaka

MhmdBarazi · Aug 29, 2023

jakaskerl

I trained a previous model, it was working properly. I transferred the files from Raspberry Pi to another PC, and even the proper model worked slowly there, that PC is really fast. I have only one class I don't think that my model is heavy there is something else.

MhmdBarazi · Aug 29, 2023

MhmdBarazi @erik Do you have an idea dear?

erik · Aug 29, 2023

MhmdBarazi If it works fast on the PC but slow on Rpi it means it's not a problem with depthai./OAK, but with the script that is running on the host (PC/RPi). I would debug the python code to find bottlenecks.

MhmdBarazi · Aug 29, 2023

erik No, the first model worked properly on RP. Both models didn't work properly on PC. The code is as simple as you gave it to me above. Could I explain the situation correctly now?

MhmdBarazi · Aug 29, 2023

I tried the camera with Depthai_SDK, it worked properly with high resolution. Just in my model, it is working like that.

MhmdBarazi · Aug 29, 2023

another issue, I couldn't get the spatial values out from this function. Sorry, I know it is a small issue but I am really new in this domain. Any help, please.

Thanks

jakaskerl · Aug 29, 2023

MhmdBarazi another issue, I couldn't get the spatial values out from this function. Sorry, I know it is a small issue but I am really new in this domain. Any help, please.

GettingSpatial is a defined as a callback function that expects a DetectionPacket as argument. You can't just call the function without any input.
This is why you specify the callback function inside oak.callback(nn.out.passthrough, GettingSpatial). This makes it so the nn packet (passthrough) gets sent to the GettingSpatial callback function as the argument whenever there is a detection present.

You can get spatials with nn.out.spatials as well.

Thanks,
Jaka

MhmdBarazi · Aug 29, 2023

jakaskerl

Nothing appeared on the console. Can you figure out what I did wrongly?
Thanks

jakaskerl · Aug 29, 2023

Hi MhmdBarazi
nn.out.spatials should replace nn.out.passthrough inside the callback. Remove the A=whatever; it won't work. If you wish to print the spatials, do so inside the callback function you have created (GettingSpatial).
Refer to:

Also please paste the code here, don't send screenshots as it is hard do recreate here.

Thanks,
Jaka

MhmdBarazi · Aug 31, 2023

Hello dear jakaskerl

Firstly, thank you so much for your efforts.

What I am trying to do is to do some calculations based on the spatial location of my detection. In the photo below, I tried to print and send the data from the function as you told me. after the first reading and calculation the program stops as you can see.

This is the code that I am using:

import numpy as np

from depthai_sdk import OakCamera, ArgsParser

import argparse

import depthai as dai

import serial

import time

if name == 'main':

Arduino_data = serial.Serial('COM3', 115200)

Arduino_data.reset_input_buffer()

dx= 27.35

dy= 123.0979

dz= 45.57

yT= 452

x0 = 0

y0 = 0

z0 = 0

Gamma=0

def GettingSpatial(packet):

while True:



    for det in packet.detections:

        spatials = det.img_detection.spatialCoordinates

        xc= spatials.x

        x0= xc + dx


    

        zc= spatials.z

        Alpha= np.arctan(yT/zc)

        CosAlpha= np.cos(Alpha)

        z0= dz + (zc \*CosAlpha)

        Gamma=np.rad2deg(np.arctan(x0/z0))     

    

    if Arduino_data.in_waiting > 0:

        data = str(Gamma)

        print(data)

        Arduino_data.write(data.encode("utf-8"))

        line = Arduino_data.readline().decode('utf-8').rstrip()

        print(line)

parser = argparse.ArgumentParser()

parser.add_argument("-conf", "--config", help="Trained YOLO json config path", default=r"C:\Users\user\Desktop\codes\320\best300.json", type=str)

args = ArgsParser.parseArgs(parser)

with OakCamera(args=args) as oak:

    stereo = oak.create_stereo(fps=8)

    color = oak.create_camera('color', fps=8)

    nn = oak.create_nn(args['config'], color, nn_type='yolo', spatial=stereo)

    oak.visualize(nn, fps=True, scale=2/3)

    oak.callback(nn.out.passthrough, GettingSpatial)  

    oak.start(blocking=True)

Thanks,

MhmdBarazi · Aug 31, 2023

I found some issues with transferring data through a USB port. This is the last code I have used:

import numpy as np

from depthai_sdk import OakCamera, ArgsParser

import argparse

import depthai as dai

import serial

import time

# CosTheta= np.cos(np.deg2rad(40))

# CosBeta= np.cos(np.deg2rad(50))

if name == 'main':

Arduino_data = serial.Serial('COM3', 115200 )

Arduino_data.reset_input_buffer()

dx= 27.35

dy= 123.0979

dz= 45.57

yT= 452

x0 = 0

y0 = 0

z0 = 0

Gamma=0

def GettingSpatial(packet):


# while True:



    for det in packet.detections:

        spatials = det.img_detection.spatialCoordinates

 

        xc= spatials.x

        x0= xc + dx

    # y0= yc

    # yc= spatials.y

    

        zc= spatials.z

        Alpha= np.arctan(yT/zc)

        CosAlpha= np.cos(Alpha)

        z0= dz + (zc \*CosAlpha)

        Gamma=np.rad2deg(np.arctan(x0/z0))

        # print(Gamma)         

        # print("")

    

        if Arduino_data.in_waiting > 0:

            data = str(Gamma)

    #     # print(data)

            Arduino_data.write(data.encode("utf-8"))

    #     # time.sleep(5)

            line = Arduino_data.readline().decode('utf-8').rstrip()

            print(line)

# parse arguments

parser = argparse.ArgumentParser()

parser.add_argument("-conf", "--config", help="Trained YOLO json config path", default=r"C:\Users\user\Desktop\codes\320\best300.json", type=str)

args = ArgsParser.parseArgs(parser)

with OakCamera(args=args) as oak:

    stereo = oak.create_stereo(fps=8)

    color = oak.create_camera('color', fps=8)

    nn = oak.create_nn(args['config'], color, nn_type='yolo', spatial=stereo)

    oak.visualize(nn, fps=True, scale=2/3)

    oak.callback(nn.out.passthrough, GettingSpatial)  

    oak.start(blocking=True)

I want to send and receive data without affecting the camera. Sometimes I can transfer some data but the camera is affected by this process. I want to know a proper way to communicate with a USB device.

Thanks

jakaskerl · Aug 31, 2023

Hi MhmdBarazi
Try running the serial in a separate thread:

import numpy as np
from depthai_sdk import OakCamera, ArgsParser
import argparse
import depthai as dai
import serial
import threading
import queue

if __name__ == '__main__':
    Arduino_data = serial.Serial('COM3', 115200)
    Arduino_data.reset_input_buffer()

    dx = 27.35
    dy = 123.0979
    dz = 45.57
    yT = 452
    x0 = 0
    y0 = 0
    z0 = 0
    Gamma = 0

    def GettingSpatial(packet):
        for det in packet.detections:
            spatials = det.img_detection.spatialCoordinates
            xc = spatials.x
            x0 = xc + dx
            zc = spatials.z
            Alpha = np.arctan(yT/zc)
            CosAlpha = np.cos(Alpha)
            z0 = dz + (zc * CosAlpha)
            Gamma = np.rad2deg(np.arctan(x0/z0))
            data_queue.put(Gamma)

    def ArduinoThread():
        while True:
            # Send data
            if not data_queue.empty():
                data = str(data_queue.get())
                Arduino_data.write(data.encode("utf-8"))

            # Receive data
            if Arduino_data.in_waiting > 0:
                line = Arduino_data.readline().decode('utf-8').rstrip()
                print(line)

    # Create a queue for communication between threads
    data_queue = queue.Queue()

    # Start the Arduino thread
    arduino_thread = threading.Thread(target=ArduinoThread)
    arduino_thread.start()

    # parse arguments
    parser = argparse.ArgumentParser()
    parser.add_argument("-conf", "--config", help="Trained YOLO json config path", default=r"C:\Users\user\Desktop\codes\320\best300.json", type=str)
    args = ArgsParser.parseArgs(parser)

    with OakCamera(args=args) as oak:
        stereo = oak.create_stereo(fps=8)
        color = oak.create_camera('color', fps=8)
        nn = oak.create_nn(args['config'], color, nn_type='yolo', spatial=stereo)
        oak.visualize(nn, fps=True, scale=2/3)
        oak.callback(nn.out.passthrough, GettingSpatial)
        oak.start(blocking=True)

Hope this helps,
Jaka

MhmdBarazi · Aug 31, 2023

jakaskerl thanks, the code ran without issues but there was no communication. I don't know the reason exactly because I checked it quickly.

MhmdBarazi · Sep 1, 2023

jakaskerl Hello dear,

Today I rechecked the code. Still no communication. What do you think the problem is? I rearranged it like this:

import numpy as np

from depthai_sdk import OakCamera, ArgsParser

import argparse

import depthai as dai

import serial

import threading

import queue

dx = 27.35

dy = 123.0979

dz = 45.57

yT = 452

x0 = 0

y0 = 0

z0 = 0

Gamma = 0

parser = argparse.ArgumentParser()

parser.add_argument("-conf", "--config", help="Trained YOLO json config path", default=r"C:\Users\user\Desktop\codes\320\best300.json", type=str)

args = ArgsParser.parseArgs(parser)

if name == 'main':

Arduino_data = serial.Serial('COM3', 115200)

Arduino_data.reset_input_buffer()

def GettingSpatial(packet):

    for det in packet.detections:

        spatials = det.img_detection.spatialCoordinates

        xc = spatials.x

        x0 = xc + dx

        zc = spatials.z

        Alpha = np.arctan(yT/zc)

        CosAlpha = np.cos(Alpha)

        z0 = dz + (zc \* CosAlpha)

        Gamma = np.rad2deg(np.arctan(x0/z0))

        data_queue.put(Gamma)

def ArduinoThread():

    while True:

        # Send data

        if not data_queue.empty():

            data = str(data_queue.get())

            Arduino_data.write(data.encode("utf-8"))

        # Receive data

        if Arduino_data.in_waiting > 0:

            line = Arduino_data.readline().decode('utf-8').rstrip()

            print(line)

# Create a queue for communication between threads

data_queue = queue.Queue()

# Start the Arduino thread

arduino_thread = threading.Thread(target=ArduinoThread)

arduino_thread.start()

with OakCamera(args=args) as oak:

    stereo = oak.create_stereo(fps=8)

    color = oak.create_camera('color', fps=8)

    nn = oak.create_nn(args['config'], color, nn_type='yolo', spatial=stereo)

    oak.visualize(nn, fps=True, scale=2/3)

    oak.callback(nn.out.passthrough, GettingSpatial)

    oak.start(blocking=True)

jakaskerl · Sep 1, 2023

Hi MhmdBarazi

Not sure, perhaps you could add some intermediate print statements to see where the problem occurs. Perhaps gamma is not correctly converted to string and therefore can't be sent. Also, try consulting GPT since I don't think this is a depthai issue anymore and might be arduino specific.

Thanks,
Jaka