Hi,
I'm using OAK-D camera.
I'd like to get the coordinates and send the value to another Raspberry Pi.
How can that be achieved?
Unable to get the coordinates and send the value to another Raspberry Pi.
Hello uk0310 ,
You can get the spatial coordinates of objects with eg. RGB & MobilenetSSD with spatial data. After streaming them back to the host (I assume in your case RPi 1), you could stream them to second RPi via web, by eg. websockets. Tons of possibilities on this end, but it really depends on your application.
Thanks, Erik
erik
Thank you for your reply.
I'd like to perform an object-detection and want to share the coordinates of the bounding-box (plus depth-information).
I could perform an object-detection and show the coordinates of the bounding-box (plus depth-information) in display by using depthai_demo.py.
but I could not retrieve the values.
If i retrieve the values, I'll send the values to another raspberrypi by using UDP communications.
Can I retrieve the values by using RGB & MobilenetSSD with spatial data?
My UDP communication program is mentioned below.
I could send a sample value by using this program.
So, I send the coordinates of the bounding-box (plus depth-information) by replacing "d" with the values.
Hello uk0310 ,
You can also retrieve these values (bounding box, spatial coordintes) from the depthai demo using callbacks.py file, see docs here - from the onNn() function. Same with the example I linked above, you will retrieve SpatialImgDetections messages that have bounding box and XYZ coordiantes inside them.
Thanks, Erik
erik
Thank you for your reply.
I could retrieve he coordinates of the bounding-box (plus depth-information) by using SpatialImgDetections.
Thank you so much!
a year later
I am new to using CV and have been playing around with it for a school project. We are currently using https://github.com/luxonis/depthai-experiments/tree/master/gen2-yolo/device-decoding with a custom trained Yolov5 model. We can get it show our objects locations, but we cannot figure out how to export the coordinates of the objects. I'm confused as to where to place:
def onNn(nn_packet, decoded_data):
pass
Any help would be greatly appreciated.
7 days later
- Edited
We are still struggling to access the depth data. We have tried going through the documentation and exploring many examples on Github. We keeping hitting walls. The goal is to have the depth data of objects exported to the terminal with a print command.
import cv2
import numpy as np
import depthai as dai
from depthai import NNData
from depthai_sdk import OakCamera, ArgsParser
from depthai_sdk.managers import PipelineManager
from depthai_sdk.classes import DetectionPacket, Detections
from depthai_sdk.visualize.configs import TextPosition
import argparse
parser = argparse.ArgumentParser()
parser.add_argument("-conf", "--config", help="Trained YOLO json config path", default='json/best.json', type=str)
args = ArgsParser.parseArgs(parser)
def decode(nn_data: NNData) -> Detections:
print(nn_data)
layer = nn_data.getFirstLayerFp16()
print(layer)
results = np.array(layer).reshape((1, 1, -1, 7))
dets = Detections(nn_data)
print(dets)
for result in results[0][0]:
if result[2] > 0.5:
dets.add(result[1], result[2], result[3:])
return dets
def callback(packet: DetectionPacket):
visualizer = packet.visualizer
num = len(packet.img_detections.detections)
#print(packet.img_detections.detections)
#print('New msgs! Number of Balls detected:', num)
#print(packet.imgFrame)
visualizer.add_text(f"Number of Objects: {num}", position=TextPosition.TOP_MID)
visualizer.draw(packet.frame)
cv2.imshow(f'frame {packet.name}', packet.frame)
with OakCamera(args=args) as oak:
color = oak.create_camera('color')
nn = oak.create_nn(args['config'], color,spatial=True, decode_fn=decode) #nn_type='yolo' decode_fn=decode
oak.visualize(nn, callback=callback)
oak.visualize(nn.out.passthrough,fps=True)
oak.start(blocking=True)5 days later
Hi @NotGrant,
Could you please try running the following script?
import argparse
import cv2
from depthai_sdk import OakCamera, ArgsParser
from depthai_sdk.classes import DetectionPacket
from depthai_sdk.visualize.configs import TextPosition
parser = argparse.ArgumentParser()
parser.add_argument("-conf", "--config", help="Trained YOLO json config path", default='json/best.json', type=str)
args = ArgsParser.parseArgs(parser)
def callback(packet: DetectionPacket):
visualizer = packet.visualizer
num = len(packet.img_detections.detections)
visualizer.add_text(f"Number of Objects: {num}", position=TextPosition.TOP_MID)
visualizer.draw(packet.frame)
cv2.imshow(f'frame {packet.name}', packet.frame)
with OakCamera(args=args) as oak:
color = oak.create_camera('color')
nn = oak.create_nn(args['config'], color, nn_type='yolo', spatial=True) # nn_type='yolo' decode_fn=decode
oak.visualize(nn, callback=callback)
oak.visualize(nn.out.passthrough, fps=True)
oak.start(blocking=True)Also please specify the path to the blob in your json config. The following configuration worked in my case:
{
"model": {
"blob": "../models/yolov5n_coco_416x416_openvino_2021.4_6shave.blob"
},
"nn_config":
{
"output_format" : "detection",
"NN_family" : "YOLO",
"input_size": "416x416",
"NN_specific_metadata" :
{
"classes" : 80,
"coordinates" : 4,
"anchors" : [10,13, 16,30, 33,23, 30,61, 62,45, 59,119, 116,90, 156,198, 373,326],
"anchor_masks" :
{
"side52" : [0,1,2],
"side26" : [3,4,5],
"side13" : [6,7,8]
},
"iou_threshold" : 0.5,
"confidence_threshold" : 0.5
}
},
"mappings":
{
"labels":
[
"person",
"bicycle",
"car",
"motorbike",
"aeroplane",
"bus",
"train",
"truck",
"boat",
"traffic light",
"fire hydrant",
"stop sign",
"parking meter",
"bench",
"bird",
"cat",
"dog",
"horse",
"sheep",
"cow",
"elephant",
"bear",
"zebra",
"giraffe",
"backpack",
"umbrella",
"handbag",
"tie",
"suitcase",
"frisbee",
"skis",
"snowboard",
"sports ball",
"kite",
"baseball bat",
"baseball glove",
"skateboard",
"surfboard",
"tennis racket",
"bottle",
"wine glass",
"cup",
"fork",
"knife",
"spoon",
"bowl",
"banana",
"apple",
"sandwich",
"orange",
"broccoli",
"carrot",
"hot dog",
"pizza",
"donut",
"cake",
"chair",
"sofa",
"pottedplant",
"bed",
"diningtable",
"toilet",
"tvmonitor",
"laptop",
"mouse",
"remote",
"keyboard",
"cell phone",
"microwave",
"oven",
"toaster",
"sink",
"refrigerator",
"book",
"clock",
"vase",
"scissors",
"teddy bear",
"hair drier",
"toothbrush"
]
}
}Thanks,
Daniil