Yes, you can. I am using latest develop version of depthai-sdk, and have limited FPS to 8:

from depthai_sdk import OakCamera, ArgsParser
import argparse

def a(packet):
    print(packet.detections)
# parse arguments
parser = argparse.ArgumentParser()
parser.add_argument("-conf", "--config", help="Trained YOLO json config path", default='model/yolo.json', type=str)
args = ArgsParser.parseArgs(parser)

with OakCamera(args=args) as oak:
    stereo = oak.stereo(fps=8)
    color = oak.create_camera('color', fps=8)
    nn = oak.create_nn(args['config'], color, nn_type='yolo', spatial=stereo)
    oak.visualize(nn, fps=True, scale=2/3)
    oak.callback(nn.out.passthrough, a)
    oak.start(blocking=True)

    erik It worked, thank you so much, just last thing, you have used a different code, the code I have uploaded doesn't work at all it means. Does it? I want to get this spatial data to use it in another code. That is why am asking. thank you so much for helping me out.

    • erik replied to this.

      MhmdBarazi I don't now, the code you used wasn't MRE and was badly formatted, so I haven't dug into it trying to debug it.

        erik Thank you, can I get spatial detection data in a string format?

        • erik replied to this.

          Hi MhmdBarazi ,
          Yes, you can:

          from depthai_sdk import OakCamera, ArgsParser
          import argparse
          import depthai as dai
          
          def a(packet):
              for det in packet.detections:
                  spatials = det.img_detection.spatialCoordinates
                  print(spatials.x, spatials.y, spatials.z)
          
          # parse arguments
          parser = argparse.ArgumentParser()
          parser.add_argument("-conf", "--config", help="Trained YOLO json config path", default='model/yolo.json', type=str)
          args = ArgsParser.parseArgs(parser)
          
          with OakCamera(args=args) as oak:
              stereo = oak.create_stereo(fps=8)
              color = oak.create_camera('color', fps=8)
              nn = oak.create_nn(args['config'], color, nn_type='yolo', spatial=stereo)
              oak.visualize(nn, fps=True, scale=2/3)
              oak.callback(nn.out.passthrough, a)
              oak.start(blocking=True)

            erik thank you so much for helping me out.

            Today, I have trained new model. then I used a new json file. but it didn't work. this error appeared:

            y:1.2648584 width:0.057771206 height:-0.26485837 is not a valid rectangle.

            [194430105190FE1200] [1.1.2] [31.289] [SpatialDetectionNetwork(8)] [error] ROI x:0.6353549 y:1.2649865 width:0.0572443 height:-0.26498652 is not a valid rectangle.

            [194430105190FE1200] [1.1.2] [31.289] [SpatialDetectionNetwork(8)] [error] ROI x:0.030717716 y:1.1823903 width:0.023574337 height:-0.18239033 is not a valid rectangle.

            And this is the json file:

            thank you again

              jakaskerl Thank you for your answer,

              Yes, I tested my model before the conversion. Actually, I converted the model using this tool: DepthAI Tools (luxonis.com). Then these were all my inputs:

              and I did the same steps as the previous one(which worked successfully)

              erik This code was perfect and worked faster than the "main_api" code. But it couldn't read my new json file properly.

                Hi MhmdBarazi
                To view the spatial coordinates from the main api, you will have to use a spatial neural network instead of the regular one. The same is done in SDK with spatial=True flag.
                SDK likely doesn't work with your model because the parsing is different, that would mean you will need to manually write a callback function to parse the NN results.

                Thanks,
                Jaka

                  jakaskerl Hello,

                  Can you give me an example code for a manually written spatial callback function?

                  Thanks,

                  #!/usr/bin/env python3

                  from pathlib import Path

                  import sys

                  import cv2

                  import depthai as dai

                  import numpy as np

                  import time

                  import os

                  '''

                  Spatial detection network demo.

                  Performs inference on RGB camera and retrieves spatial location coordinates: x,y,z relative to the center of depth map.

                  '''

                  # Get argument first

                  nnBlobPath = str((os.path.dirname(os.path.abspath("file")) / Path('/home/apakgrup/depthai-python/examples/spatial/best_openvino_2022.1_7shave.blob')).resolve().absolute())

                  if len(sys.argv) > 1:

                  nnBlobPath = sys.argv[1]

                  if not Path(nnBlobPath).exists():

                  import sys
                  
                  raise FileNotFoundError(f'Required file/s not found, please run "{sys.executable} install_requirements.py"')

                  # MobilenetSSD label texts

                  labelMap = ["fire"]

                  syncNN = True

                  # Create pipeline

                  pipeline = dai.Pipeline()

                  # Define sources and outputs

                  camRgb = pipeline.create(dai.node.ColorCamera)

                  spatialDetectionNetwork = pipeline.create(dai.node.MobileNetSpatialDetectionNetwork)

                  monoLeft = pipeline.create(dai.node.MonoCamera)

                  monoRight = pipeline.create(dai.node.MonoCamera)

                  stereo = pipeline.create(dai.node.StereoDepth)

                  xoutRgb = pipeline.create(dai.node.XLinkOut)

                  xoutNN = pipeline.create(dai.node.XLinkOut)

                  xoutDepth = pipeline.create(dai.node.XLinkOut)

                  xoutRgb.setStreamName("rgb")

                  xoutNN.setStreamName("detections")

                  xoutDepth.setStreamName("depth")

                  # Properties

                  camRgb.setPreviewSize(640,640)

                  camRgb.setResolution(dai.ColorCameraProperties.SensorResolution.THE_1080_P)

                  camRgb.setInterleaved(False)

                  camRgb.setColorOrder(dai.ColorCameraProperties.ColorOrder.BGR)

                  monoLeft.setResolution(dai.MonoCameraProperties.SensorResolution.THE_400_P)

                  monoLeft.setCamera("left")

                  monoRight.setResolution(dai.MonoCameraProperties.SensorResolution.THE_400_P)

                  monoRight.setCamera("right")

                  # Setting node configs

                  stereo.setDefaultProfilePreset(dai.node.StereoDepth.PresetMode.HIGH_DENSITY)

                  # Align depth map to the perspective of RGB camera, on which inference is done

                  stereo.setDepthAlign(dai.CameraBoardSocket.CAM_A)

                  stereo.setSubpixel(True)

                  stereo.setOutputSize(monoLeft.getResolutionWidth(), monoLeft.getResolutionHeight())

                  spatialDetectionNetwork.setBlobPath(nnBlobPath)

                  spatialDetectionNetwork.setConfidenceThreshold(0.5)

                  spatialDetectionNetwork.input.setBlocking(False)

                  spatialDetectionNetwork.setBoundingBoxScaleFactor(0.5)

                  spatialDetectionNetwork.setDepthLowerThreshold(100)

                  spatialDetectionNetwork.setDepthUpperThreshold(5000)

                  # Linking

                  monoLeft.out.link(stereo.left)

                  monoRight.out.link(stereo.right)

                  camRgb.preview.link(spatialDetectionNetwork.input)

                  if syncNN:

                  spatialDetectionNetwork.passthrough.link(xoutRgb.input)

                  else:

                  camRgb.preview.link(xoutRgb.input)

                  spatialDetectionNetwork.out.link(xoutNN.input)

                  stereo.depth.link(spatialDetectionNetwork.inputDepth)

                  spatialDetectionNetwork.passthroughDepth.link(xoutDepth.input)

                  # Connect to device and start pipeline

                  with dai.Device(pipeline) as device:

                  # Output queues will be used to get the rgb frames and nn data from the outputs defined above
                  
                  previewQueue = device.getOutputQueue(name="rgb", maxSize=4, blocking=False)
                  
                  detectionNNQueue = device.getOutputQueue(name="detections", maxSize=4, blocking=False)
                  
                  depthQueue = device.getOutputQueue(name="depth", maxSize=4, blocking=False)
                  
                  startTime = time.monotonic()
                  
                  counter = 0
                  
                  fps = 0
                  
                  color = (255, 255, 255)
                  
                  while True:
                  
                      inPreview = previewQueue.get()
                  
                      inDet = detectionNNQueue.get()
                  
                      depth = depthQueue.get()
                  
                      counter+=1
                  
                      current_time = time.monotonic()
                  
                      if (current_time - startTime) > 1 :
                  
                          fps = counter / (current_time - startTime)
                  
                          counter = 0
                  
                          startTime = current_time
                  
                      frame = inPreview.getCvFrame()
                  
                      depthFrame = depth.getFrame() # depthFrame values are in millimeters
                  
                      depth_downscaled = depthFrame[::4]
                  
                      min_depth = np.percentile(depth_downscaled[depth_downscaled != 0], 1)
                  
                      max_depth = np.percentile(depth_downscaled, 99)
                  
                      depthFrameColor = np.interp(depthFrame, (min_depth, max_depth), (0, 255)).astype(np.uint8)
                  
                      depthFrameColor = cv2.applyColorMap(depthFrameColor, cv2.COLORMAP_HOT)
                  
                      detections = inDet.detections
                  
                      # If the frame is available, draw bounding boxes on it and show the frame
                  
                      height = frame.shape[0]
                  
                      width  = frame.shape[1]
                  
                      for detection in detections:
                  
                          roiData = detection.boundingBoxMapping
                  
                          roi = roiData.roi
                  
                          roi = roi.denormalize(depthFrameColor.shape[1], depthFrameColor.shape[0])
                  
                          topLeft = roi.topLeft()
                  
                          bottomRight = roi.bottomRight()
                  
                          xmin = int(topLeft.x)
                  
                          ymin = int(topLeft.y)
                  
                          xmax = int(bottomRight.x)
                  
                          ymax = int(bottomRight.y)
                  
                          cv2.rectangle(depthFrameColor, (xmin, ymin), (xmax, ymax), color, 1)
                  
                          # Denormalize bounding box
                  
                          x1 = int(detection.xmin \* width)
                  
                          x2 = int(detection.xmax \* width)
                  
                          y1 = int(detection.ymin \* height)
                  
                          y2 = int(detection.ymax \* height)
                  
                          try:
                  
                              label = labelMap[detection.label]
                  
                          except:
                  
                              label = detection.label
                  
                          cv2.putText(frame, str(label), (x1 + 10, y1 + 20), cv2.FONT_HERSHEY_TRIPLEX, 0.5, 255)
                  
                          cv2.putText(frame, "{:.2f}".format(detection.confidence\*100), (x1 + 10, y1 + 35), cv2.FONT_HERSHEY_TRIPLEX, 0.5, 255)
                  
                          cv2.putText(frame, f"X: {int(detection.spatialCoordinates.x)} mm", (x1 + 10, y1 + 50), cv2.FONT_HERSHEY_TRIPLEX, 0.5, 255)
                  
                          cv2.putText(frame, f"Y: {int(detection.spatialCoordinates.y)} mm", (x1 + 10, y1 + 65), cv2.FONT_HERSHEY_TRIPLEX, 0.5, 255)
                  
                          cv2.putText(frame, f"Z: {int(detection.spatialCoordinates.z)} mm", (x1 + 10, y1 + 80), cv2.FONT_HERSHEY_TRIPLEX, 0.5, 255)
                  
                          cv2.rectangle(frame, (x1, y1), (x2, y2), (255, 0, 0), cv2.FONT_HERSHEY_SIMPLEX)
                  
                      cv2.putText(frame, "NN fps: {:.2f}".format(fps), (2, frame.shape[0] - 4), cv2.FONT_HERSHEY_TRIPLEX, 0.4, (255,255,255))
                  
                      cv2.imshow("depth", depthFrameColor)
                  
                      cv2.imshow("preview", frame)
                  
                      if cv2.waitKey(1) == ord('q'):
                  
                          break

                  I ran this code from depth-ai examples then the same error appeared. this is my last blob and json file:

                  any help please?

                  • erik replied to this.

                    erik I installed the latest version(https://github.com/luxonis/depthai-python/releases/tag/v2.22.0.0) still give me this error:

                    [194430105190FE1200] [1.1.2] [12.735] [XLinkOut(6)] [error] Message has too much metadata (618343B) to serialize. Maximum is 51200B. Dropping message

                    [194430105190FE1200] [1.1.2] [13.450] [SpatialDetectionNetwork(1)] [error] ROI x:0.59461975 y:0.67529297 width:0.006866455 height:0 is not a valid rectangle.

                    [194430105190FE1200] [1.1.2] [13.450] [SpatialDetectionNetwork(1)] [error] ROI x:0.683197 y:0.78881836 width:0 height:0.012207031 is not a valid rectangle.

                    [194430105190FE1200] [1.1.2] [13.453] [SpatialDetectionNetwork(1)] [error] ROI x:0.6419296 y:0.7573242 width:0.010299683 height:0 is not a valid rectangle.

                    [194430105190FE1200] [1.1.2] [13.456] [SpatialDetectionNetwork(1)] [error] ROI x:0.5257492 y:0.5644531 width:0.031723022 height:0 is not a valid rectangle.

                    [194430105190FE1200] [1.1.2] [13.457] [SpatialDetectionNetwork(1)] [error] ROI x:0.500618 y:0.54003906 width:0.0045318604 height:0 is not a valid rectangle.

                    [194430105190FE1200] [1.1.2] [13.457] [SpatialDetectionNetwork(1)] [error] ROI x:0.5377655 y:0.49804688 width:0.0063171387 height:0 is not a valid rectangle.

                    The weirdest part of this is that the first model is still working. But after that nothing worked

                      jakaskerl Hello,

                      I installed this on my raspberry pi:

                      apakgrup@raspberrypi:~ $ git clone https://github.com/luxonis/depthai.git

                      Right now, these are the installed versions :

                      depthai==2.22.0.0

                      depthai-pipeline-graph==0.0.5

                      depthai-sdk==1.12.1

                      Still not workingโ€ฆ

                      Thanks,

                      @MhmdBarazi what script and arguments are you using? It might be a problem if you have best.xml / best.bin, as if that's the case, it will use already compiled model (as it only checks name), instead of compiling a new one. So i'd suggest changing the names of xml/bin (and update to new names inside your .json)