LLcicC
- a month ago
- Joined Jul 26, 2024
- 1 best answer
- Edited
We'll check. In the meantime, do you think that the camera with broken RGB sensor must be returned and substituted?
- Edited
Hi lovro ,
Concerning the first camera, we tried all the things you suggested without success. Moreover, depthai-viewer reports that it cannot connect to the device.
For the second camera we followed the Guide. Where are crash reports stored?
We implemented an autoreconnect system but the device was not reachable.Thanks
Hello,
I have two cameras of the above model with different issues.
MxId 14442C10A1E2F2D600
The RGB sensor is not working. All the scripts accessing the stream get stuck.
Other Camera (no MxId available at the moment)
Suddenly, all the scripts accessing the camera returned XLINK device not found.
Thanks in advance
- In FPS Drop
Hi Jaka, tried with setPreviewSize and the output is still not aligned
- In FPS Drop
- Edited
Hi Jaka,
with
monoRight.setResolution(dai.MonoCameraProperties.SensorResolution.THE_720_P)
monoLeft.setResolution(dai.MonoCameraProperties.SensorResolution.THE_720_P)
camRgb.setVideoSize(640, 360)
stereo.setOutputSize(640, 360)output is not aligned.
Tried adding setPreviewSize or isp scale for the resolution of the color camera but couldn't make it work.
- In FPS Drop
Hi Jaka,
Good to know, thanks! Now I’m facing another issue: when I set the same resolution for setVideoSize, setOutputSize, and setResolution, RGB and DEPTH are aligned. However, that is not the case if I choose a smaller size for setOutputSize and setVideoSize.
Example:
using resolution
monoRight.setResolution(dai.MonoCameraProperties.SensorResolution.THE_720_P) monoLeft.setResolution(dai.MonoCameraProperties.SensorResolution.THE_720_P)
with
camRgb.setVideoSize(1280, 720)
stereo.setOutputSize(1280, 720)
I get a reasonable alignment of RGB and DEPTH:
But with
monoRight.setResolution(dai.MonoCameraProperties.SensorResolution.THE_720_P) monoLeft.setResolution(dai.MonoCameraProperties.SensorResolution.THE_720_P)
and
camRgb.setVideoSize(320, 240)
stereo.setOutputSize(320, 240)
I get
- Best Answerset by jakaskerl
Hi Jaka,
Thaanks! It turns out all I had to do was press the spacebar to continue (!). Wouldn't it be helpful to have a message saying so? With the video, I thought the person was just closing the window in the interface, and the tutorial doesn't mention it. The script itself says to press the spacebar to continue in the first step, but (maybe it's just me!) I didn’t think to do that after the step of taking the images. It’s not just after the first pop-up; you need to press the spacebar multiple times until the calibration is fully done. Also, I assume it's set up this way so you can review and validate each pop-up before continuing, so wouldn’t it be nice if the tutorial explained that process too—like what to look for during each step?
Thanks again.
- In FPS Drop
Indeed setting camRgb.setVideoSize(1280, 720) and stereo.setOutputSize(1280, 720) works. How do these methods work though? I mean, are the frames being taken with the resolution that was set and then getting resized in the camera after all the post-processing operations? And I guess I could put any reasonable size here, if color and depth are in alignment at least.
Hi Jaka,
I have waited both with the window opened, and after having closed the window. It gets stuck.
I’m using command python3 calibrate.py -s 3.6 –board OAK-D-POE-W-97 -nx 15 -ny 8, with charuco board for 28” screen and depthai version 2.28.0.0.
getEepromData() gives product name OAK-D-W-POE-97 and board name NG9097.
- In FPS Drop
It is not unsupported as changing the code the resolution changes as well.
Yes, the board is 15x8 (not the same of the first image). My bad, there is the pop window and then it gets stuck.
Using 28” screen, and command python3 calibrate.py -s 3.3 –board OAK-D-POE-W-97 -nx 15 -ny 8, it looks alright but then the calibrate.py gets stuck right after the three “INTRINSIC CALIBRATIONS”.
- In FPS Drop
Hi Jaka,
Even when we set rgb and depth to the same resolution, of 1280x720, using either setSize or ispScale depending on the case, we get inconsistent results. For example, using rgb_depth_aligned.py (from depthai-python/examples/StereoDepth/rgb_depth_aligned.py at main · luxonis/depthai-python · GitHub) with one camera we get the expected resolution but with another one the rgb frame comes in full HD regardless of setDepthAlign being set.
- In FPS Drop
Nice, your example reproduce problem 2: the setFps does not set Fps, since you set it to 30 and got 5 with POE. Any ideas how to get a higher fps in this case?
Hi jakaskerl
Repo version: the lastest from GitHub - luxonis/depthai: DepthAI Python API utilities, examples, and tutorials., I guess its v3.6.1
Depthai version: also the lastest, 2.28.0.0
Command to call calibrate: python3 calibrate.py -s 3.3 –board OAK-D-PRO-W-POE -nx 7 -ny 4
Charuco board: the 28” screen one as per link in the tutorial Calibration (luxonis.com)
I also tried with an older version of depthai (2.24), different cameras, and printing the charuco board instead of displaying it on the screen.
- In FPS Drop
Hi jakaskerl ,
Yes, fps means frames per second and I’m computing it with the time it takes to get both rgb and depth frames, like:start_time with time.time()
get message with qSync.get() and frames with msg.getCvFrame()
compute fps as 1/(time.time()-s_time))
I’m trying to set the fps to reasonable values such as 10, 20 or 30, using cam.setFps(FPS).
However, I’m getting frames arriving at very high (100, 300) or very low (1, 0.5) rates.
- In FPS Drop
Hi there,
I need to get both color and depth streams, aligned and in sync, with medium resolution and a good fps. I'm running the same code, with depthai 2.28.0.0, with different cameras and getting inconsistent outcomes. The main problem is the drop of fps - understood as the time to get both frames. Specifically, I have three issues:
1. For different cameras, same setting (i.e. literally same computer and same cable), I get frames with a resolution different than the one that was set.
2. The setFPS does not set FPS.
3. For different settings, the FPS drops from like 300 to 1, with literally the same code and the same camera (same object, not version), but different computer/cables.
I am always using depthai 2.28 and cables cat 6 A. I tested the network, checked the devices, updated firmware and tested different scripts from github depthai-experiments; including bandwidth and latency tests as per https://github.com/luxonis/depthai-experiments/tree/master/random-scripts.
Partial code:
#################################################################
# CREATE nodes
monoLeft = pipeline.create(dai.node.MonoCamera)
monoRight = pipeline.create(dai.node.MonoCamera)
stereo = pipeline.create(dai.node.StereoDepth)
rgb = pipeline.create(dai.node.Camera)
sync = pipeline.create(dai.node.Sync)
xoutGrp = pipeline.create(dai.node.XLinkOut)
xoutGrp.setStreamName("xout")
# set Properties RGB
rgbCamSocket = dai.CameraBoardSocket.CAM_A
rgb.setBoardSocket(rgbCamSocket)
rgb.setSize(1280, 720)
rgb.setFps(FPS)
# set Properties LEFT
monoLeft.setResolution(dai.MonoCameraProperties.SensorResolution.THE_720_P)
monoLeft.setCamera("left") #monoLeft.setBoardSocket(dai.CameraBoardSocket.CAM_B)
monoLeft.setFps(FPS)
# set Properties RIGHT
monoRight.setResolution(dai.MonoCameraProperties.SensorResolution.THE_720_P)
monoRight.setCamera("right") #monoRight.setBoardSocket(dai.CameraBoardSocket.CAM_C)
monoRight.setFps(FPS)
# set Properties STEREO
stereo.setDepthAlign(rgbCamSocket)
stereo.initialConfig.setConfidenceThreshold(confidenceThreshold)
stereo.setLeftRightCheck(lr_check)
stereo.setSubpixel(subpixel)
stereo.setExtendedDisparity(extended_disparity)
stereo.initialConfig.setMedianFilter(dai.StereoDepthProperties.MedianFilter.KERNEL_7x7) # to reduce noise and smoothen the depth map
# set Properties SYNC
sync.setSyncThreshold(timedelta(milliseconds=100))
# Linking
monoLeft.out.link(stereo.left)
monoRight.out.link(stereo.right)
stereo.depth.link(sync.inputs["depth"])
rgb.video.link(sync.inputs["rgb"])
sync.out.link(xoutGrp.input)
if alpha is not None:
stereo.setAlphaScaling(alpha)
rgb.setCalibrationAlpha(alpha)
#####################################################
with dai.Device(pipeline) as device:
qSync = device.getOutputQueue(name="xout", maxSize=10, blocking=True)
while True:
s_time=time.time()
msgGrp = qSync.get()
for name, msg in msgGrp:
frame = msg.getCvFrame()
if name == "depth":
depth_frame = frame
depth_clock = msg.getTimestamp()
depth_bits = depth_frame.nbytes*8
depth_shape = depth_frame.shape
if name == "rgb":
rgb_frame = frame
rgb_clock = msg.getTimestamp()
rgb_bits = rgb_frame.nbytes*8
rgb_shape = rgb_frame.shape
print("Delta depth-rgb {} ms".format( (depth_clock - rgb_clock).total_seconds()*1000))
print("Bits depth {} and Bits rgb {}".format(depth_bits, rgb_bits))
print("Shape depth {} and Bits rgb {}".format(depth_shape, rgb_shape))
print("***fps sync : ",1/(time.time()-s_time))
#################################################################
Outcomes:
######################### Outcome 1 #########################
OAK-D PoE AF
Available camera sensors: {<CameraBoardSocket.CAM_B: 1>: 'OV9282', <CameraBoardSocket.CAM_A: 0>: 'IMX378', <CameraBoardSocket.CAM_C: 2>: 'OV9282'}Product name: OAK-D PoE AF, board name DM2088IMU type: BNO086
firmware version: 3.9.9, embedded firmware version: 3.9.9
USB speed: UsbSpeed.UNKNOWN
Delta depth-rgb 0.546 ms
Bits depth 33177600 and Bits rgb 49766400
Shape depth (1080, 1920) and rgb (1080, 1920, 3)
fps sync : 227.5679019044002
######################### Outcome 2 #########################
OAK-D-W-POE-97
Available camera sensors: {<CameraBoardSocket.CAM_B: 1>: 'OV9282', <CameraBoardSocket.CAM_A: 0>: 'OV9782', <CameraBoardSocket.CAM_C: 2>: 'OV9282'}Product name: OAK-D-W-POE-97, board name NG9097IMU type: BNO086,
firmware version: 3.9.9, embedded firmware version: 3.9.9
USB speed: UsbSpeed.UNKNOWN
Delta depth-rgb 95.834 ms
Bits depth 14745600 and Bits rgb 22118400
Shape depth (720, 1280) and rgb (720, 1280, 3)
fps sync : 269.03810134701735
######################## Outcome 3 #########################
OAK-D-PRO-W-POE
Available camera sensors: {<CameraBoardSocket.CAM_B: 1>: 'OV9282', <CameraBoardSocket.CAM_A: 0>: 'OV9782', <CameraBoardSocket.CAM_C: 2>: 'OV9282'}Product name: OAK-D-PRO-W-POE, board name NG9097IMU type: BNO086,
firmware version: 3.9.9, embedded firmware version: 3.9.9
USB speed: UsbSpeed.UNKNOWN
Delta depth-rgb 96.121 ms
Bits depth 14745600 and Bits rgb 22118400
Shape depth (720, 1280) and rgb (720, 1280, 3)
fps sync : 583.7583855254002
######################### Outcome 4 #########################
OAK-D-PRO-W-POE-97
Available camera sensors: {<CameraBoardSocket.CAM_B: 1>: 'OV9282', <CameraBoardSocket.CAM_A: 0>: 'OV9782', <CameraBoardSocket.CAM_C: 2>: 'OV9282'}
Product name: OAK-D-PRO-W-POE-97, board name NG9097IMU type: BNO086,
firmware version: 3.9.9, embedded firmware version: 3.9.9
USB speed: UsbSpeed.UNKNOWN
Delta depth-rgb 45.458999999999996 ms
Bits depth 14745600 and Bits rgb 22118400
Shape depth (720, 1280) and rgb (720, 1280, 3)
fps sync : 3.3333285649685727
######################### Outcome 5 #########################
OAK-D-PRO-W-POE (not the same as Outcome 3)
Available camera sensors: {<CameraBoardSocket.CAM_B: 1>: 'OV9282', <CameraBoardSocket.CAM_A: 0>: 'IMX378', <CameraBoardSocket.CAM_C: 2>: 'OV9282'}Product name: OAK-D-PRO-W-POE, board name NG9097IMU type: BNO086,
firmware version: 3.9.9, embedded firmware version: 3.9.9
USB speed: UsbSpeed.UNKNOWN
Delta depth-rgb 0.856 ms
Bits depth 33177600 and Bits rgb 49766400
Shape depth (1080, 1920) and rgb (1080, 1920, 3)
fps sync : 0.6608250229397115
Any idea?
Thanks