OAK-D-IoT-75 with raspberry pi.

erik · May 1, 2022

Hello msee19018 , Could you share the pipeline, so we can debug it locally as well? Could you also try forcing USB2 mode?
Thanks, Erik

Mmsee19018 · May 11, 2022

Hi Erik, I am getting this error on raspberry pi 4. It does not matter if i use usb2Mode or usb3Mode it gives same error. I have pasted the error below. Btw the code works okay with my PC and even with jetson nano as host but it does not work with RPi 4. I even installed ubuntu on RPi but that also gives same error. I have also pasted the code below. It is long code and has a couple of script nodes. Can you please suggest any solution.

`import cv2
import depthai as dai
import numpy as np
import marshal
import time
import logging
import os
import requests

g_time=0
cur_time=''
cur_local=''
cur_visit=''

real_path = os.path.realpath(file)
dir_path = os.path.dirname(real_path)
LOGFILE = f"{dir_path}/test.log"
logger = logging.getLogger('log_app')

logger.setLevel(logging.DEBUG)
fh = logging.FileHandler(LOGFILE)
formatter = logging.Formatter('%(levelname)s - %(message)s')
fh.setFormatter(formatter)
logger.addHandler(fh)

quarter_time_break=False
prev_msb=0
sus_msb=0
msb_changed=False
distinct_msbs=[]
prev_digit_time=0
prev_time=''
prev_local=''
prev_visiting=''
const_time_counter=0
game_status='unknown'
game_period='unknown'
clock_type=''
sus_time=''
sus_local=''
sus_visiting=''
local_changed=False
visiting_changed=False
seconds=False
time_changed=False
clock_condition=True
seconds_counter=0
minutes_counter=0
frame_num=0
message_string=''
whitelist=set('0123456789')
time_bb_cord=[0.3734,0.6837,0.6437,0.8487]
local_bb_cord=[0.3117,0.3775,0.4523,0.52625]
visit_bb_cord=[0.5891,0.37,0.7367,0.5225]

Create pipeline

def Manip_Frame(pipeline,region_bb_cord):
manip_time = pipeline.create(dai.node.ImageManip)
manip_time.initialConfig.setCropRect(region_bb_cord[0],region_bb_cord[1],region_bb_cord[2],region_bb_cord[3])
manip_time.setKeepAspectRatio(False)
manip_time.initialConfig.setResize(100,32)
return manip_time
def NN_node(pipeline,path):
nn = pipeline.create(dai.node.NeuralNetwork)
nn.setBlobPath(path)
return nn

pipeline = dai.Pipeline()
model_path = 'crnn_99_soft_no_norm_4.blob'

cam = pipeline.create(dai.node.MonoCamera)
cam.setBoardSocket(dai.CameraBoardSocket.RIGHT)
cam.setFps(6)
cam.setResolution(dai.MonoCameraProperties.SensorResolution.THE_800_P)

manip_time=Manip_Frame(pipeline,time_bb_cord)
cam.out.link(manip_time.inputImage)

manip_local=Manip_Frame(pipeline,local_bb_cord)
cam.out.link(manip_local.inputImage)

manip_visit=Manip_Frame(pipeline,visit_bb_cord)
cam.out.link(manip_visit.inputImage)

script=pipeline.create(dai.node.Script)

script.inputs['time_frame'].setBlocking(False)

script.inputs['time_frame'].setQueueSize(1)

script.inputs['local_frame'].setBlocking(False)

script.inputs['local_frame'].setQueueSize(1)

script.inputs['visit_frame'].setBlocking(False)

script.inputs['visit_frame'].setQueueSize(1)

manip_time.out.link(script.inputs['time_frame'])
manip_local.out.link(script.inputs['local_frame'])
manip_visit.out.link(script.inputs['visit_frame'])

script.setScript("""
import marshal
f_list={}
while True:
t_frame=node.io['time_frame'].get()
l_frame=node.io['local_frame'].get()
v_frame=node.io['visit_frame'].get()
if t_frame is not None and 'time' not in f_list.keys():
t_f_num=t_frame.getSequenceNum()
f_list['time']=t_f_num
if l_frame is not None and 'local' not in f_list.keys():
l_f_num=t_frame.getSequenceNum()
f_list['local']=l_f_num
if v_frame is not None and 'visit' not in f_list.keys():
v_f_num=t_frame.getSequenceNum()
f_list['visit']=v_f_num

if len(f_list.keys())==3 and f_list['time']==f_list['local'] and f_list['time']==f_list['visit']:
	# x_serial=marshal.dumps(f_list)
	# b=Buffer(len(x_serial))
	# b.setData(x_serial)
	f_list.clear()
	node.io['script_out'].send(t_frame)
	node.io['script_out1'].send(l_frame)
	node.io['script_out2'].send(v_frame)
	# node.warn(str(len(t_frame.getData())))
	# node.warn(str(l_frame.getWidth()))
	# node.warn(str(v_frame.getWidth()))
	# node.warn('camera is okay')
	""")

recog_nn=NN_node(pipeline,model_path)

script.outputs['script_out'].link(recog_nn.input)
script.outputs['script_out1'].link(recog_nn.input)
script.outputs['script_out2'].link(recog_nn.input)

decoder_script=pipeline.create(dai.node.Script)
recog_nn.out.link(decoder_script.inputs['log_probs'])

decoder_script.setScript("""

import marshal
label2char={1: '0', 2: '1', 3: '2', 4: '3', 5: '4', 6: '5', 7: '6', 8: '7', 9: '8', 10: '9', 11: '.', 12: ':'}
def get_strings(nndata):

num=nndata.getSequenceNum()

outname=nndata.getAllLayerNames()[0]
data=nndata.getLayerFp16(outname)
raw_preds=[]
for i in range(24):
log_probs=data[i13:i13+13]
raw_preds.append(log_probs.index(max(log_probs)))
results=[]
previous=None
for l in raw_preds:
if l != previous:
results.append(l)
previous = l
results = [l for l in results if l != 0]

results = [label2char[l] for l in results]

f_results=''
for r in range(len(results)):
f_results=f_results+label2char[results[r]]
return f_results
outs=[]
while True:
time_=node.io['log_probs'].get()

node.warn(time_.getSequenceNum())

node.warn(str(len(time_.getData())))

results=get_strings(time_)

node.warn(str(len(results)))

if len(results)==0:
results='none'

print()

if len(outs)==0:
outs.append(results)
elif len(outs)>0:
if results=='none':
outs.append(results)
elif results!=outs[-1]:
outs.append(results)
if len(outs)==3:
x_serial = marshal.dumps(outs)
b = Buffer(len(x_serial))
b.setData(x_serial)
outs.clear()
node.io['f_time'].send(b)

node.warn('decoding is okay')

""")

nn_Out = pipeline.create(dai.node.XLinkOut)
nn_Out.setStreamName("recogs1")
decoder_script.outputs['f_time'].link(nn_Out.input)

with dai.Device(pipeline) as device:
nn_queue1 = device.getOutputQueue(name="recogs1", maxSize=4, blocking=False)
i=0
j=0
while True:
nn_out1 = nn_queue1.get()
if nn_out1 is not None:
score_dict=marshal.loads(nn_out1.getData())

print(score_dict)

if i<30:
i=i+1
continue
if len(score_dict[0])>3:
result_time=score_dict[0]
local=score_dict[1]
visit=score_dict[2]
elif len(score_dict[1])>3:
result_time=score_dict[1]
local=score_dict[2]
visit=score_dict[0]
elif len(score_dict[2])>3:
result_time=score_dict[2]
local=score_dict[0]
visit=score_dict[1]
if result_time=='none' or local =='none' or visit=='none':
j=j+1
print('it is here')
if j>30:
message_string='Game is not active'
print(message_string)
fh.close()
j=0
break
continue
if clock_condition:
if len(result_time)==4:
seconds_counter=seconds_counter+1
minutes_counter=0
continue
elif len(result_time)==5:
minutes_counter=minutes_counter+1
seconds_counter=0
if seconds_counter==3:
seconds=True
clock_condition=False
elif minutes_counter==3:
seconds=False
clock_condition=False
t_msb=int(result_time[0])
if len(distinct_msbs)==0:
distinct_msbs.append(t_msb)

		if msb_changed:
			if t_msb==sus_msb:
				msb_changed=False
				if distinct_msbs[-1]==0 and t_msb==5:
					seconds=True
				distinct_msbs.append(t_msb)
			elif t_msb!=sus_msb:
				msb_changed=True
				sus_msb=t_msb
		else:
			if t_msb!=prev_msb:
				sus_msb=t_msb
				msb_changed=True
		if seconds:
			c_time=result_time[0]+result_time[1]
		else:
			c_time=result_time

		ds_time=''.join(filter(whitelist.__contains__,c_time))
		local=''.join(filter(whitelist.__contains__,local))
		visiting=''.join(filter(whitelist.__contains__,visit))

		if len(ds_time)!=0 and len(local)!=0 and len(visiting)!= 0:
			# print(len(ds_time),len(local),len(visiting))
			game_status='active'
			j=0
			if time_changed:
				if c_time==sus_time:
					const_time_counter=0
					time_changed=False
					# distinct_frames.append(ds_time)
					if game_status=='active' and game_period=='unknown':
						game_period='on_going'
					elif game_status=='active' and game_period=='stopped' and not quarter_time_break:
						game_period='on_going'
						g_time=time.time()
						print('Game is on_going')
						if seconds:
							message_string='Game started again! Time '+c_time+' Local Team Score = '+local+' Visitng Team Score = '+visiting
							g_time=time.time()
							print(message_string)
							logger.info(message_string)
						else:
							message_string='Game started again! Time '+c_time+' Local Team Score = '+local+' Visitng Team Score = '+visiting
							g_time=time.time()
							print(message_string)
							logger.info(message_string)
				elif c_time!=sus_time:
					time_changed=True
					sus_time=c_time
					const_time_counter=0
					frame_num=frame_num+1
					continue
			else:
				if c_time==prev_time:
					const_time_counter=const_time_counter+1
				elif c_time!=prev_time:
					sus_time=c_time
					time_changed=True
					frame_num=frame_num+1
					continue

			if local_changed:
				if local==sus_local:
					local_changed=False
					if frame_num>10:
						message_string='Score of Local Team is changed: Time '+c_time+' Local Team Score = '+local+' Visitng Team Score = '+visiting
						g_time=time.time()
						print(message_string)
						logger.info(message_string)
				elif local!=sus_local:
					local_changed=True
					sus_local=local
			else:
				if local!=prev_local:
					sus_local=local
					local_changed=True

			if visiting_changed:
				if visiting==sus_visiting:
					visiting_changed=False
					if frame_num>10:
						message_string='Score of Visiting Team is changed: Time '+c_time+' Local Team Score = '+local+' Visitng Team Score = '+visiting
						g_time=time.time()
						print(message_string)
						logger.info(message_string)
				elif visiting!=sus_visiting:
					visinting_changed=True
					sus_visiting=visiting
			else:
				if visiting!=prev_visiting:
					sus_visiting=visiting
					visiting_changed=True
			if const_time_counter>=12 and game_status=='active' and not quarter_time_break:
				if result_time=='00:0' and seconds:
					quarter_time_break=True
					game_period='stopped'
					message_string='It is Quarter Time Out: Score of Local Team is '+local+' and Score of Visiting Team is '+visiting
					g_time=time.time()
					print(message_string)
					logger.info(message_string)
			if const_time_counter>=12 and game_status=='active' and game_period=='stopped':
				if result_time=='08:00' and quarter_time_break:
					seconds=False
					quarter_time_break=False
					game_period='on_going'
					g_time=time.time()
					#node.warn('Game is About to start again')
			elif const_time_counter>=12 and game_status=='active' and game_period=='on_going' and quarter_time_break==False:
				const_time_counter=0
				game_period='stopped'
				if not time=='08:00':
					if seconds:
						message_string='Game is stopped: Time: '+result_time+'s Score of Local Team is '+local+' and Score of Visiting Team is '+visiting
						g_time=time.time()
						print(message_string)
						logger.info(message_string)
					else:
						message_string='Game is stopped: Time: '+result_time+' Score of Local Team is '+local+' and Score of Visiting Team is '+visiting
						g_time=time.time()
						print(message_string)
						logger.info(message_string)
			if g_time-time.time()>10 and quarter_time_break==False:
				message_string= 'Time: '+result_time+' Score of Local Team is '+local+' and Score of Visiting Team is '+visiting
				g_time=time.time()
				logger.info(message_string)
			prev_msb=t_msb
			prev_time=c_time
			prev_local=local
			prev_visiting=visiting
			frame_num=frame_num+1

`

Mmsee19018 · May 11, 2022

Hi Erik, It ran good on jetson nano first time but now even that is giving same error. But the same code works good on my PC, it does not matter how many times i run it. I could not post the .py file here so the above code is not very presentable. Some times it gives this error too. segmentation fault.

erik · May 12, 2022

Hi msee19018 , that's interesting, it might be an issue in firmware. Could you provide minimal reproducible code that crashes on RPi/Jetson, so we can take a look at it?
Thanks, Erik

Mmsee19018 · May 12, 2022

Hi Erik Thanks for your response. Right now even a simple code where i am just displaying video from Right Mono Camera is not working. It is giving both of the above errors on Jetson Nano. Below is the given code.

import cv2

pipeline = dai.Pipeline()
cam = pipeline.create(dai.node.MonoCamera)
cam.setBoardSocket(dai.CameraBoardSocket.RIGHT)
cam.setFps(6)
cam.setResolution(dai.MonoCameraProperties.SensorResolution.THE_800_P)

xoutRight = pipeline.create(dai.node.XLinkOut)
xoutRight.setStreamName('right')
cam.out.link(xoutRight.input)

with dai.Device(pipeline) as device:
    qRight = device.getOutputQueue(name="right", maxSize=4, blocking=False)
    while True:
        inRight = qRight.get()
        if inRight is not None:
            cv2.imshow("right", inRight.getCvFrame())
        if cv2.waitKey(1) == ord('q'):
            break`

erik · May 12, 2022

Hi msee19018 , what's the USB mode used (this example logs it in terminal)? Are you using USB2 cable? Could you also try the same code with the same cable/OAK on a laptop/PC?
Thanks, Erik

Mmsee19018 · May 12, 2022

I am using the cable that i got with the device. It is usb3 and when i run the above mentioned code it prints out Usb speed: SUPER on both my PC and jetson nano. Even this script crashes after printing for a little time.

erik · May 13, 2022

msee19018 Interesting.. I suspect it's the low FPS issue, could you try commenting out that line? If it is, I will also report it to firmware devs so they can have a look at it.
Thanks, Erik

Mmsee19018 · May 16, 2022

Hi Erik, I tried using high FPS but even that did not solve the problem. I tried the same code again with Jetson Nano after rebooting the Nano and to my surprise it started working. Now the situation is the same code works with Nano on oak-d device and it does not work with RPi 4. It gives same error segmentation fault and bus error. The code I am referring to is the long code that i pasted above. It has a script node then a neural network node and then finally again a script node to post process networks output. Basically I am using CRNN to recognize digits from pre-defined positions from frames of Mono camera. Can you please suggest a solution?
Another thing that might help you is if i connect oak-d with my PC and then again connect it with my Nano. It starts giving the same errors i think this is what happened last time that is why it stopped working with Nano. But now I am connecting it only with Nano from the beginning and it is working fine. But the same solution does not work with RPi. I finally need to deliver solution with RPi. So please look into it?

erik · May 17, 2022

Hello msee19018 ,
From our previous communication, I am not sure if you are using the preconfigured RPi image that we provide. As there are quite a few things you have to be mindful of when installing all the required packages so DepthAI works as expected. Could you please confirm that?
Thanks, Erik

Mmsee19018 · May 18, 2022

Hi Erik, I am using these pre configured images. I would try this now can you tell which one I should download? I Have oak-d-iot-75?

erik · May 18, 2022

Hi msee19018 , Probably the latest OAK_CM4_POE version, so V8.
Thanks, Erik

Mmsee19018 · May 22, 2022

Hi Erik I did that, I burned the above mentioned image on my SD card. It worked for a couple of times but then again it started giving the error. I have pasted the pic of error below. It is different from above errors(bus error and segmentation fault). So at least that problem is gone. Can you please look what is causing this problem?

erik · May 23, 2022

Hi msee19018 , could you first try running depthai_demo application and copy the log? My first assumption would be that you are using USB2 instead of USB3 cable, which causes the app to crash.
Thanks, Erik

Mmsee19018 · May 23, 2022

No that is not the case. The cable is same which I was using earlier. It prints speed: Super if I use rgb_preview example, that you mentioned above. I will use depthai_demo. Another thing is as i mentioned above it worked a couple of times on same code.

erik · May 23, 2022

Hello msee19018 ,
from my understanding even a basic mono streaming example crashes on RPi with preconfigured RPi OS img and with USB3 connectivity?
Thanks, Erik

Mmsee19018 · May 23, 2022

Here are some more explanation and photos of error. As i have mentioned above it works some times too. I have found out that if I turn on my RPi (after shutting it down for sometime) the script starts working and it works fine for some time before it starts giving the error after that nothing works on RPi even the simplest scripts. I do not know why it works after resting RPi for some time. I was working on it yesterday after it started giving me the error I turned off RPi and I turned it on just now and it again worked for like an hour and then again it started giving the error and right now nothing works on RPi and the RPi is giving a pretty detailed error I have pasted its photos below. It was giving same error yesterday too. Please have a look.

erik · May 23, 2022

Hello msee19018 , Which version of the depthai library are you currently using? It looks like it's a special version from all the verbose logging.

Mmsee19018 · May 24, 2022

It is the same version that came with the prebuilt image. Its version is 2.13.3.0.dev. I have also attached the screen shoot.

erik · May 24, 2022

Hi msee19018 , Could you try updating depthai to the latest version? 2.13 is a bit old, and we might have fixed this issue already in updates.
THanks, Erik