jakaskerl Thank you for your answer,

Yes, I tested my model before the conversion. Actually, I converted the model using this tool: DepthAI Tools (luxonis.com). Then these were all my inputs:

and I did the same steps as the previous one(which worked successfully)

erik This code was perfect and worked faster than the "main_api" code. But it couldn't read my new json file properly.

    Hi MhmdBarazi
    To view the spatial coordinates from the main api, you will have to use a spatial neural network instead of the regular one. The same is done in SDK with spatial=True flag.
    SDK likely doesn't work with your model because the parsing is different, that would mean you will need to manually write a callback function to parse the NN results.

    Thanks,
    Jaka

      jakaskerl Hello,

      Can you give me an example code for a manually written spatial callback function?

      Thanks,

      #!/usr/bin/env python3

      from pathlib import Path

      import sys

      import cv2

      import depthai as dai

      import numpy as np

      import time

      import os

      '''

      Spatial detection network demo.

      Performs inference on RGB camera and retrieves spatial location coordinates: x,y,z relative to the center of depth map.

      '''

      # Get argument first

      nnBlobPath = str((os.path.dirname(os.path.abspath("file")) / Path('/home/apakgrup/depthai-python/examples/spatial/best_openvino_2022.1_7shave.blob')).resolve().absolute())

      if len(sys.argv) > 1:

      nnBlobPath = sys.argv[1]

      if not Path(nnBlobPath).exists():

      import sys
      
      raise FileNotFoundError(f'Required file/s not found, please run "{sys.executable} install_requirements.py"')

      # MobilenetSSD label texts

      labelMap = ["fire"]

      syncNN = True

      # Create pipeline

      pipeline = dai.Pipeline()

      # Define sources and outputs

      camRgb = pipeline.create(dai.node.ColorCamera)

      spatialDetectionNetwork = pipeline.create(dai.node.MobileNetSpatialDetectionNetwork)

      monoLeft = pipeline.create(dai.node.MonoCamera)

      monoRight = pipeline.create(dai.node.MonoCamera)

      stereo = pipeline.create(dai.node.StereoDepth)

      xoutRgb = pipeline.create(dai.node.XLinkOut)

      xoutNN = pipeline.create(dai.node.XLinkOut)

      xoutDepth = pipeline.create(dai.node.XLinkOut)

      xoutRgb.setStreamName("rgb")

      xoutNN.setStreamName("detections")

      xoutDepth.setStreamName("depth")

      # Properties

      camRgb.setPreviewSize(640,640)

      camRgb.setResolution(dai.ColorCameraProperties.SensorResolution.THE_1080_P)

      camRgb.setInterleaved(False)

      camRgb.setColorOrder(dai.ColorCameraProperties.ColorOrder.BGR)

      monoLeft.setResolution(dai.MonoCameraProperties.SensorResolution.THE_400_P)

      monoLeft.setCamera("left")

      monoRight.setResolution(dai.MonoCameraProperties.SensorResolution.THE_400_P)

      monoRight.setCamera("right")

      # Setting node configs

      stereo.setDefaultProfilePreset(dai.node.StereoDepth.PresetMode.HIGH_DENSITY)

      # Align depth map to the perspective of RGB camera, on which inference is done

      stereo.setDepthAlign(dai.CameraBoardSocket.CAM_A)

      stereo.setSubpixel(True)

      stereo.setOutputSize(monoLeft.getResolutionWidth(), monoLeft.getResolutionHeight())

      spatialDetectionNetwork.setBlobPath(nnBlobPath)

      spatialDetectionNetwork.setConfidenceThreshold(0.5)

      spatialDetectionNetwork.input.setBlocking(False)

      spatialDetectionNetwork.setBoundingBoxScaleFactor(0.5)

      spatialDetectionNetwork.setDepthLowerThreshold(100)

      spatialDetectionNetwork.setDepthUpperThreshold(5000)

      # Linking

      monoLeft.out.link(stereo.left)

      monoRight.out.link(stereo.right)

      camRgb.preview.link(spatialDetectionNetwork.input)

      if syncNN:

      spatialDetectionNetwork.passthrough.link(xoutRgb.input)

      else:

      camRgb.preview.link(xoutRgb.input)

      spatialDetectionNetwork.out.link(xoutNN.input)

      stereo.depth.link(spatialDetectionNetwork.inputDepth)

      spatialDetectionNetwork.passthroughDepth.link(xoutDepth.input)

      # Connect to device and start pipeline

      with dai.Device(pipeline) as device:

      # Output queues will be used to get the rgb frames and nn data from the outputs defined above
      
      previewQueue = device.getOutputQueue(name="rgb", maxSize=4, blocking=False)
      
      detectionNNQueue = device.getOutputQueue(name="detections", maxSize=4, blocking=False)
      
      depthQueue = device.getOutputQueue(name="depth", maxSize=4, blocking=False)
      
      startTime = time.monotonic()
      
      counter = 0
      
      fps = 0
      
      color = (255, 255, 255)
      
      while True:
      
          inPreview = previewQueue.get()
      
          inDet = detectionNNQueue.get()
      
          depth = depthQueue.get()
      
          counter+=1
      
          current_time = time.monotonic()
      
          if (current_time - startTime) > 1 :
      
              fps = counter / (current_time - startTime)
      
              counter = 0
      
              startTime = current_time
      
          frame = inPreview.getCvFrame()
      
          depthFrame = depth.getFrame() # depthFrame values are in millimeters
      
          depth_downscaled = depthFrame[::4]
      
          min_depth = np.percentile(depth_downscaled[depth_downscaled != 0], 1)
      
          max_depth = np.percentile(depth_downscaled, 99)
      
          depthFrameColor = np.interp(depthFrame, (min_depth, max_depth), (0, 255)).astype(np.uint8)
      
          depthFrameColor = cv2.applyColorMap(depthFrameColor, cv2.COLORMAP_HOT)
      
          detections = inDet.detections
      
          # If the frame is available, draw bounding boxes on it and show the frame
      
          height = frame.shape[0]
      
          width  = frame.shape[1]
      
          for detection in detections:
      
              roiData = detection.boundingBoxMapping
      
              roi = roiData.roi
      
              roi = roi.denormalize(depthFrameColor.shape[1], depthFrameColor.shape[0])
      
              topLeft = roi.topLeft()
      
              bottomRight = roi.bottomRight()
      
              xmin = int(topLeft.x)
      
              ymin = int(topLeft.y)
      
              xmax = int(bottomRight.x)
      
              ymax = int(bottomRight.y)
      
              cv2.rectangle(depthFrameColor, (xmin, ymin), (xmax, ymax), color, 1)
      
              # Denormalize bounding box
      
              x1 = int(detection.xmin \* width)
      
              x2 = int(detection.xmax \* width)
      
              y1 = int(detection.ymin \* height)
      
              y2 = int(detection.ymax \* height)
      
              try:
      
                  label = labelMap[detection.label]
      
              except:
      
                  label = detection.label
      
              cv2.putText(frame, str(label), (x1 + 10, y1 + 20), cv2.FONT_HERSHEY_TRIPLEX, 0.5, 255)
      
              cv2.putText(frame, "{:.2f}".format(detection.confidence\*100), (x1 + 10, y1 + 35), cv2.FONT_HERSHEY_TRIPLEX, 0.5, 255)
      
              cv2.putText(frame, f"X: {int(detection.spatialCoordinates.x)} mm", (x1 + 10, y1 + 50), cv2.FONT_HERSHEY_TRIPLEX, 0.5, 255)
      
              cv2.putText(frame, f"Y: {int(detection.spatialCoordinates.y)} mm", (x1 + 10, y1 + 65), cv2.FONT_HERSHEY_TRIPLEX, 0.5, 255)
      
              cv2.putText(frame, f"Z: {int(detection.spatialCoordinates.z)} mm", (x1 + 10, y1 + 80), cv2.FONT_HERSHEY_TRIPLEX, 0.5, 255)
      
              cv2.rectangle(frame, (x1, y1), (x2, y2), (255, 0, 0), cv2.FONT_HERSHEY_SIMPLEX)
      
          cv2.putText(frame, "NN fps: {:.2f}".format(fps), (2, frame.shape[0] - 4), cv2.FONT_HERSHEY_TRIPLEX, 0.4, (255,255,255))
      
          cv2.imshow("depth", depthFrameColor)
      
          cv2.imshow("preview", frame)
      
          if cv2.waitKey(1) == ord('q'):
      
              break

      I ran this code from depth-ai examples then the same error appeared. this is my last blob and json file:

      any help please?

      • erik replied to this.

        erik I installed the latest version(https://github.com/luxonis/depthai-python/releases/tag/v2.22.0.0) still give me this error:

        [194430105190FE1200] [1.1.2] [12.735] [XLinkOut(6)] [error] Message has too much metadata (618343B) to serialize. Maximum is 51200B. Dropping message

        [194430105190FE1200] [1.1.2] [13.450] [SpatialDetectionNetwork(1)] [error] ROI x:0.59461975 y:0.67529297 width:0.006866455 height:0 is not a valid rectangle.

        [194430105190FE1200] [1.1.2] [13.450] [SpatialDetectionNetwork(1)] [error] ROI x:0.683197 y:0.78881836 width:0 height:0.012207031 is not a valid rectangle.

        [194430105190FE1200] [1.1.2] [13.453] [SpatialDetectionNetwork(1)] [error] ROI x:0.6419296 y:0.7573242 width:0.010299683 height:0 is not a valid rectangle.

        [194430105190FE1200] [1.1.2] [13.456] [SpatialDetectionNetwork(1)] [error] ROI x:0.5257492 y:0.5644531 width:0.031723022 height:0 is not a valid rectangle.

        [194430105190FE1200] [1.1.2] [13.457] [SpatialDetectionNetwork(1)] [error] ROI x:0.500618 y:0.54003906 width:0.0045318604 height:0 is not a valid rectangle.

        [194430105190FE1200] [1.1.2] [13.457] [SpatialDetectionNetwork(1)] [error] ROI x:0.5377655 y:0.49804688 width:0.0063171387 height:0 is not a valid rectangle.

        The weirdest part of this is that the first model is still working. But after that nothing worked

          jakaskerl Hello,

          I installed this on my raspberry pi:

          apakgrup@raspberrypi:~ $ git clone https://github.com/luxonis/depthai.git

          Right now, these are the installed versions :

          depthai==2.22.0.0

          depthai-pipeline-graph==0.0.5

          depthai-sdk==1.12.1

          Still not working…

          Thanks,

          @MhmdBarazi what script and arguments are you using? It might be a problem if you have best.xml / best.bin, as if that's the case, it will use already compiled model (as it only checks name), instead of compiling a new one. So i'd suggest changing the names of xml/bin (and update to new names inside your .json)

            erik Hello dear,

            Thank you so much it worked, but unfortunately there is a new problem:

            the window just shows an image, the image that you are seeing know doesn't change. there is no video no motion.

            Thanks,

              Hi MhmdBarazi
              In the code below, you are using a callback. Could you recheck that it works correctly. Maybe try removing it to see if it fixes the problem.

              Thanks,
              Jaka

              Hello dear@"jakaskerl"#p11449,

              I tried from a different device, with and without callback. Still the same problem

              Update, There is a non-continuous motion in the window(sometimes it freezes) as well as it is too slow.

                Hi MhmdBarazi
                Upon testing the model locally, I believe it's to computationally expensive. Just as you have experienced above, I am also only getting about 0.5 FPS. This is not the case if I'm using other models.

                Thoughts?
                Jaka

                  I am relatively new in this domain unfortunately. But I tested the previous model on the other PC, it gave me a similar performance. What can you suggest to me jakaskerl ?

                  Thanks

                    Hi MhmdBarazi
                    The problem is OAK device processing speed, that's why the results are the same regardless of the host you are using. I believe you would have to scale down the model (set input to 300x300 instead of 640x640, which is very high) to achieve faster inference, perhaps change the framework (iirc yolo6 runs the fastest)..

                    Thanks,
                    Jaka